Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use cache for both poetry workflow runs #133

Merged
merged 19 commits into from
Jul 23, 2023
Merged

use cache for both poetry workflow runs #133

merged 19 commits into from
Jul 23, 2023

Conversation

andrewj-brown
Copy link
Member

by caching poetry dependencies, we should hopefully significantly speed up the testing and the typing runs.

Theoretically the "fastest" option is to install pyright/pytest on their own, but then we have two places where dependencies are stored, instead of the single-source-of-truth being poetry.

@andrewj-brown
Copy link
Member Author

andrewj-brown commented Jul 1, 2023

...first cache hit, looks like the net saving is ~5 seconds? Poetry accounts for the vast majority of that setup time :(

@49Indium 49Indium added the enhancement update an existing command or cog for some new functionality label Jul 2, 2023
Copy link
Member

@JamesDearlove JamesDearlove left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hate to be that person, but there's a much simpler way: https://github.com/actions/setup-python#caching-packages-dependencies

Imo if you're looking for improving testing run times, I think that combining the static check + unit test actions would be better. After all, they are both essentially testing the code itself. Black can remain separate, as it can run in parallel to these and doesn't require poetry.

@andrewj-brown
Copy link
Member Author

andrewj-brown commented Jul 13, 2023

I hate to be that person, but there's a much simpler way: https://github.com/actions/setup-python#caching-packages-dependencies

Imo if you're looking for improving testing run times, I think that combining the static check + unit test actions would be better. After all, they are both essentially testing the code itself. Black can remain separate, as it can run in parallel to these and doesn't require poetry.

There was something about unable to pass certain parameters into poetry when using setup-python's cache, but I'll give it a look. If we use setup-python's cache it might even cache the actual poetry install, which would be significantly faster.

@andrewj-brown
Copy link
Member Author

andrewj-brown commented Jul 14, 2023

Yeah OK so I can make it cache the actual poetry install, which will be the fastest option (saves ~19s of poetry install on cache hit) but completely locks us out of using setup-python's autocaching because it requires us actually caching the .poetry folder... which is the opposite of your suggestion 💀

I'd prefer to keep tests & typing separate so its easier to see what actually failed on a particular PR, although we don't really have enough tests for that to be a massive problem (yet). Also, because they're separate jobs, it won't really save time - they run in parallel anyway.

@andrewj-brown
Copy link
Member Author

Alright, we're now successfully caching poetry and the poetry dependencies.

It looks like the test workflow has gone from ~40s to ~20s, and the typecheck workflow has gone from ~40s to ~30s. (I'm unable to work out how the heck those numbers work, given we've eliminated the same steps from both workflows, but whatever.)

I don't believe it's possible to cache anything else, or if its even worth it considering the setup is now 13s of basically just "download the cache files".

Copy link
Member

@JamesDearlove JamesDearlove left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah look, I don't think you're gonna improve much more with the action run times here. I think that even below a minute is still very solid.

Still though, good work

@andrewj-brown andrewj-brown merged commit 41b1bd4 into main Jul 23, 2023
3 checks passed
@andrewj-brown andrewj-brown deleted the cache-actions branch July 23, 2023 02:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement update an existing command or cog for some new functionality
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants