Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Keep all times, show stddev in HTML; better runner loop #2

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

belluzj
Copy link
Collaborator

@belluzj belluzj commented Mar 26, 2021

@simoncozens I noticed when it was time for me to push that I worked on similar things to you, ie. the idea that the timed thing shouldn't be a random bash script but a single command.

I also did a few other things that you might find useful:

  • keep all the times and show stddev in the HTML
  • make the timed command run in a "work directory" that gets wiped between each run
  • a better run loop that starts by enumerating all combinations and then reports progress (also to debug it you can make it stop after the first combination)

I didn't try merging just now because I'm off for today but I'll rebase next week on top of your latest work.

@simoncozens
Copy link
Collaborator

I’m not convinced about enumerating all combinations first. I notice a lot of time is spent shuffling around Python installations. I want to try reordering the list so that the project/version is in the outer loop and the setup script is only done once for each version, then the scenario is in the inner loop.

simoncozens added a commit that referenced this pull request Mar 26, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants