Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Faster CI? #510

Open
wentasah opened this issue Oct 20, 2024 · 1 comment · May be fixed by #519
Open

Faster CI? #510

wentasah opened this issue Oct 20, 2024 · 1 comment · May be fixed by #519

Comments

@wentasah
Copy link
Contributor

Hi @lopsided98 ,

if I understand correctly, you build all ROS packages on GitHub runners and aarch64-linux packages are built via qemu. This must take very long time. At our university, I've got a few powerful servers, whose power I could donate to building ROS packages. Specifically, I have an 80-core aarch64-linux machine and a 32-core x86_64-linux machine. Would you be interested in using these to speed-up development in this repo?

I see several possibilities how to use them. The simplest one, which can be working very quickly, is that the servers can be used as nix remote builders. Either for CI running on GitHub, or even for you personal Hydra (I think you have one somewhere). Another option would be to run Hydra (or other CI system) on those servers to and integrate it in this repo.

Let me know what do you think and we can then discuss details such as security, trustworthiness etc.

@wentasah wentasah linked a pull request Oct 27, 2024 that will close this issue
@lopsided98
Copy link
Owner

We do need better CI, since right now the GitHub Actions build is almost completely broken. I think the builders are running out of memory. I can try further reducing the number of jobs, but then I think it will still run into the maximum build time limit.

I do build this overlay on my private Hydra instance, but I'd like to keep it private since it builds a lot of other stuff as well.

Adding remote builders to the GitHub Actions CI is a possibility, but it would put you and anyone else with admin access to the build servers in a position of trust, and even if this is clearly documented, that doesn't mean that existing users would necessarily be comfortable with it.

In fact, I should probably make it more clear in the documentation that by using the binary cache, you are effectively giving me code execution on your machine (even root privileges in some cases). There may be a false sense of security because the packages are built on GitHub Actions, but this is moot because I control the Cachix private key.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants