Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

properly handle signals (SIGINT, SIGTERM) to terminate the server correctly #274

Open
andy108369 opened this issue Aug 21, 2022 · 1 comment
Assignees

Comments

@andy108369
Copy link
Contributor

andy108369 commented Aug 21, 2022

A shell script is running the binary, when the container receives SIGTERM (say on Pod termination or docker stop / kill with the default SIGTERM, or Ctrl+C - SIGINT) the shell script does not forward these signals to the binary.

You can quickly test and see SIGTERM is not working, just issue docker [compose] stop -t600 node it won't do anything, until it times out and issues the SIGKILL which is not stopping the binary gracefully.

Cosmos SDK understands SIGTERM & SIGINT for graceful server termination.

Signal handling can be done directly in the shell script, in cosmos-omnibus that would be both run.sh and snapshot.sh.

We can reuse the signal handling from one of my projects I've been working on - self-update.

Workaround

Get into the container and then kill <pid-of-cosmos-binary> (issues SIGTERM by default).

@andy108369 andy108369 self-assigned this Aug 21, 2022
andy108369 added a commit to andy108369/cosmos-omnibus that referenced this issue Jun 27, 2023
@andy108369 andy108369 reopened this Jun 27, 2023
@andy108369
Copy link
Contributor Author

andy108369 commented Jun 28, 2023

could probably leverage the wait -n method (it waits for any process ./process1 &, ./process2 &, ... to exit in the main script) https://docs.docker.com/config/containers/multi-service_container/
if it helps in any way after modifying the run.sh & snapshot.sh scripts to handle the signals.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant