-
Notifications
You must be signed in to change notification settings - Fork 56
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
various unrelated small fixes and tweaks #1752
Commits on Nov 3, 2024
-
crates/data-plane-controller: adjust defaults and log about DNS propa…
…gation Use some concurrency by default, and increase the heartbeat interval. Log when we're waiting for DNS propagation.
Configuration menu - View commit details
-
Copy full SHA for 5d8444b - Browse repository at this point
Copy the full SHA 5d8444bView commit details -
flowctl raw bearer-logs: add --since parameter with 1 hour default
If there are many historical logs in the stream, you can get a lot of output when reading from the beginning of time and the operator is usually only interested in what's happening right now.
Configuration menu - View commit details
-
Copy full SHA for 678a3ab - Browse repository at this point
Copy the full SHA 678a3abView commit details -
go/runtime: ensure we destroy a SQLite store on Open error
We were not properly tearning down these instances, which lead to lingering open RocksDB instances and open file descriptors to removed files, preventing disk from being reclaimed.
Configuration menu - View commit details
-
Copy full SHA for c64e5c5 - Browse repository at this point
Copy the full SHA c64e5c5View commit details -
gazette: make Router synchronous and lazy, and clear Read header
Simplify Router by starting a Channel immediately, and instead emphasize RPC timeouts rather than connection timeouts. There are a variety of conditions where connection timeouts don't really work, such as when a server has bound its port but is not actively serving it. Currently this causes tonic to block indefinitely. Also, Channel embeds a reconnection behavior if the transport is broken, so it's already the case that Channels are connecting in the background and we should expect to see corresponding delays. So, have a single behavior for Channels: they start immediately but may have connection delay in the background. - Back out connection-readiness route selection in favor of random balancing. - Distinguish local vs non-local Channels: in the future, we should use this to implement selective compression over non-local Channels. Also, upon an error a Read stream must clear its ReadRequest Header. The rationale is that the current Header could point to a Gazette broker which has permanently stopped, and connections will time out repeatedly. Clearing the Header causes route discovery to restart using the base service address.
Configuration menu - View commit details
-
Copy full SHA for 747ecc3 - Browse repository at this point
Copy the full SHA 747ecc3View commit details
Commits on Nov 4, 2024
-
derive-sqlite: improve transform stub generation
Lead with an identity-transform example SELECT JSON($flow_document) Continue to emit an example selecting individual fields, but semantically disable it and also remove root document projections, because these are never desired.
Configuration menu - View commit details
-
Copy full SHA for 876bc9c - Browse repository at this point
Copy the full SHA 876bc9cView commit details
Commits on Nov 5, 2024
-
gazette/runtime: use HTTP/2 keep-alive intervals
HTTP/2 keep-alive sends a PING frame every interval, and fails the connection of the peer doesn't respond in time. This verifies the end-to-end health of the HTTP/2 transport and catches issues like servers which have bound sockets but aren't actively listening. Also using HTTP/2 keep-alive when connecting to local containers. We've observed that `podman` can fail in ways that leave the reactor believing it has an established connection to flow-connector-init, even though the container has failed and the network namespace has been torn down.
Configuration menu - View commit details
-
Copy full SHA for c703fa6 - Browse repository at this point
Copy the full SHA c703fa6View commit details