Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add GitHub workflow for continuous benchmarking. #304

Merged
merged 5 commits into from
May 30, 2024

Commits on May 23, 2024

  1. Add GitHub workflow for continuous benchmarking.

    The workflow runs on-demand by commenting `/benchmark` on any pull
    request. It runs a few iterations of both the base branch and the PR and
    posts the results back to the pull request as a comment. Additionally,
    detailed plots are available as images in the workflow artifacts.
    Unfortunately GitHub doesn't have a way for actions to upload images in
    comments.
    
    At the moment the workflow runs two benchmarks, one at a 10k population
    size and 10k time steps, and second one at 1M population size and 1k
    time steps.
    
    The workflow currently takes on the order of 1 hour to run. We can
    easily tweak the set of benchmarks in the future if we want to run more
    scenarios, or conversly if we want to reduce the total time.
    
    There is no long-term tracking of benchmark results yet. Doing so is
    tricky as the performance of the underlying CI is unlikely to be stable
    long-term.
    plietar committed May 23, 2024
    Configuration menu
    Copy the full SHA
    3cb593d View commit details
    Browse the repository at this point in the history
  2. Remove raw data upload.

    It was merged into touchstone upstream.
    plietar committed May 23, 2024
    Configuration menu
    Copy the full SHA
    142d857 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    14f1cbc View commit details
    Browse the repository at this point in the history
  4. Downgrade ubuntu back to 22.04.

    I'm getting issues with 24.04 where pak installs some system packages
    with apt-get, some system services are upgraded and restarted and the CI
    runner is canceled. Not sure what the reason is, but 22.04 doesn't have
    this problem.
    plietar committed May 23, 2024
    Configuration menu
    Copy the full SHA
    45f8c3f View commit details
    Browse the repository at this point in the history

Commits on May 30, 2024

  1. Add R_GC_MEM_GROW

    plietar committed May 30, 2024
    Configuration menu
    Copy the full SHA
    89b7199 View commit details
    Browse the repository at this point in the history