-
Notifications
You must be signed in to change notification settings - Fork 9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: beacon depends on validator and bridge #28
Conversation
docker-compose.yml
Outdated
depends_on: | ||
- celestia-network-bridge | ||
- reth | ||
celestia-network-validator: | ||
condition: service_healthy | ||
celestia-network-bridge: | ||
condition: service_started | ||
reth: | ||
condition: service_started |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think this makes any logical difference as the celestia-network-bridge
doesn't start until the validator is "healthy" anyway.
If you wanted to add a health check to the bridge node, you could try using some lightweight method like /network_head
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point. I tried to add a health check using node.Info or p2p.Info but both required an admin token. Looks like /network_head
only needs a read auth token so seems a tad easier.
Ref: https://node-rpc-docs.celestia.org/?version=v0.20.4
Created celestiaorg/celestia-node#4007 so celestia-node exposes an un authenticated status endpoint. In the meantime DevOps shared a workaround that they use:
exec [sh -c AUTH_TOKEN=$(celestia bridge auth read --node.store=/home/celestia 2>/dev/null); curl --max-time 0.5 -s -X POST -H "Content-type: application/json" -H "Accept: application/json" -H "Authorization: Bearer $AUTH_TOKEN" -d '{"id":1,"jsonrpc":"2.0","method":"header.SyncWait","params":[]}' "http://localhost:26658"; if [ "$?" -eq 0 ]; then exit 0; else echo "Catching up"; exit 1; fi ]
so I'll try something like that
A couple of findings:
|
Update the celestia bridge node is starting prematurely. If I retry starting it after the validator has been up for some time, it works. However, on initial launch it doesn't work and fails to start the RPC server. The healthcheck on the validator doesn't seem to be granular enough. |
docker-compose.yml
Outdated
test: ["CMD-SHELL", "/bin/sh -c 'BLOCK_HEIGHT=$(curl -sf http://localhost:26657/status | jq -r .result.sync_info.latest_block_height); if [ \"$$BLOCK_HEIGHT\" -gt 1 ]; then exit 0; else echo \"Block height too low: $$BLOCK_HEIGHT\"; exit 1; fi'"] | ||
interval: 1s | ||
timeout: 5s | ||
retries: 5 | ||
retries: 20 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[note for reviewers] this makes Docker report this service healthy only if the block height is > 1
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can you please push your branch to the repo because i can't fetch it unless i set your fork as upstream 🙏🏻
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ran this a couple times and worked for me every time
Closes #24
The whole stack now reliably starts