Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create Dockerfile to run test_skim.py inside locally. #572

Closed
wants to merge 1 commit into from

Conversation

gwpl
Copy link

@gwpl gwpl commented Jul 19, 2024

Usage:
docker build -f Dockerfile -q . | xargs -I % docker run % Allows to easily run locally python tests, like it's run in ci.yml, in isolated container for reproducible and local testing.

Usage:
docker build -f Dockerfile -q . | xargs -I % docker run %
Allows to easily run locally python tests, like it's run in ci.yml, in isolated container for reproducible and local testing.
@yazgoo
Copy link
Collaborator

yazgoo commented Nov 7, 2024

would it be possible to use this docker container in gh action ?

IMO otherwise it duplicates logic which makes it hard to maintain.

@gwpl
Copy link
Author

gwpl commented Nov 7, 2024

I hope it's ok that in those busy times I generated answer for you:


LLM Agent:

Yes, it is possible and actually quite common to reuse Docker containers in GitHub Actions to avoid duplication and improve maintainability.

How to Approach This

  1. Create a Reusable Dockerfile:

    • Design the Dockerfile in a way that makes it useful both locally and in CI (GitHub Actions).
    • Ensure it has all the dependencies, setup scripts, and configurations required to run the project consistently in both environments.
  2. Use the Docker Container in GitHub Actions:

    • In the GitHub Actions workflow, you can use the Docker container by specifying the docker build and docker run commands as part of your CI pipeline, or directly reference the Dockerfile in the container option of a job:
    jobs:
      build:
        runs-on: ubuntu-latest
        container:
          image: docker://<your-image> # You can either pull from Docker Hub or build from the local Dockerfile
        steps:
          - name: Checkout code
            uses: actions/checkout@v2
          - name: Build and Test
            run: |
              # Commands to build/test within the container
              make build
              make test
  3. Building the Image Locally vs. in GitHub Actions:

    • You can build the image locally for development, and also build it in GitHub Actions if needed. If you want to avoid building twice, you could publish the Docker image to a registry like Docker Hub or GitHub Container Registry and pull it in both environments.
  4. Avoiding Redundant Code and Config:

    • By using the same Dockerfile and setup, you ensure that any changes to dependencies or configurations only need to be made in one place, reducing the risk of mismatched environments between local and CI/CD.

Advantages

  • Consistency: Your local environment mirrors the CI environment, which helps to catch issues early.
  • Reduced Maintenance: One Dockerfile handles both local development and CI needs, avoiding duplicate logic and effort.

Summary

Using a single Dockerfile for both local development and GitHub Actions is not only possible but also a best practice when aiming for consistency and maintainability in CI/CD pipelines.

@yazgoo
Copy link
Collaborator

yazgoo commented Nov 7, 2024

I don't doubt this is doable.
could you do this in your PR ?

we are just starting owning this code and I don't want to add duplicated logic when possible.

@LoricAndre
Copy link
Contributor

One issue with this is that we can't use it to run tests on MacOS, it is useful in itself but using it in the CI adds too much overhead

@LoricAndre
Copy link
Contributor

Adding something like this could still be quite useful for local development and testing, but:

  • Your image isn't working on the current test suite since one of them uses xxd, which is not included. Either rewrite the test to achieve the same without it or add xxd to the image
  • Ubuntu images are huge, something like a debian-slim or an alpine-based would be far better
  • We should rather use an image with cargo built in upstream rather than installing it ourselves with rustup, which takes quite a lot of time

@LoricAndre LoricAndre force-pushed the master branch 3 times, most recently from 6b0fab9 to e649d08 Compare November 26, 2024 17:01
@LoricAndre
Copy link
Contributor

I am closing this, since the e2e have been rewritten in rust: #629

@LoricAndre LoricAndre closed this Dec 1, 2024
@gwpl
Copy link
Author

gwpl commented Dec 1, 2024

Thank you!

I just checked e2e directory and I see rust code controlling Tmux for uni testing!
Only took fast look and already seems like promising piece of opensource!
Thank you for skim and infra (e2e) to keep it in shape ! :)

@gwpl
Copy link
Author

gwpl commented Dec 1, 2024

Do you want me to make new Pull Request to run New Rust Intrastructure end to end tests in Docker file?

It would help with hacking skim and testing skim without messing local tmux sessions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants