Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix build & deployment to Docker Hub #4110

Merged
merged 6 commits into from
Oct 12, 2024
Merged

Fix build & deployment to Docker Hub #4110

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
abd7773
[automation] Auto-update linters version, help and documentation
nvuillam Oct 11, 2024
2d57898
[MegaLinter] Apply linters fixes
nvuillam Oct 11, 2024
9261c0b
trvy
nvuillam Oct 12, 2024
16e2ef3
Deploy also on docker hub for alpha
nvuillam Oct 12, 2024
fad9506
Login to docker hub
nvuillam Oct 12, 2024
0c99a55
Update Beta generation
nvuillam Oct 12, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
374 changes: 194 additions & 180 deletions .automation/generated/linter-helps.json
View file
Open in desktop

Large diffs are not rendered by default.

22 changes: 11 additions & 11 deletions .automation/generated/linter-versions.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
"bash-exec": "5.2.26",
"bicep_linter": "0.30.23",
"black": "24.10.0",
"cfn-lint": "1.16.0",
"cfn-lint": "1.16.1",
"checkmake": "0.2.0",
"checkov": "3.2.257",
"checkstyle": "10.18.2",
Expand All @@ -18,14 +18,14 @@
"coffeelint": "5.2.11",
"cpplint": "2.0.0",
"csharpier": "0.29.2",
"cspell": "8.14.4",
"cspell": "8.15.1",
"dartanalyzer": "0.0.0",
"detekt": "1.23.7",
"devskim": "1.0.33",
"djlint": "1.35.2",
"dockerfilelint": "1.8.0",
"dotenv-linter": "3.3.0",
"dotnet-format": "8.0.108",
"dotnet-format": "8.0.110",
"dustilock": "1.2.0",
"editorconfig-checker": "3.0.3",
"eslint": "8.57.1",
Expand Down Expand Up @@ -65,7 +65,7 @@
"php": "7.4.26",
"php-cs-fixer": "3.64.0",
"phpcs": "3.10.3",
"phplint": "9.5.2",
"phplint": "9.5.3",
"phpstan": "1.12.6",
"pmd": "7.6.0",
"powershell": "7.4.2",
Expand All @@ -76,11 +76,11 @@
"psalm": "Psalm.5.26.1@",
"puppet-lint": "4.2.4",
"pylint": "3.3.1",
"pyright": "1.1.383",
"pyright": "1.1.384",
"raku": "2020.10",
"remark-lint": "14.0.2",
"revive": "1.4.0",
"roslynator": "0.9.0.0",
"roslynator": "0.9.1.0",
"rst-lint": "1.4.0",
"rstcheck": "6.2.4",
"rstfmt": "0.0.14",
Expand All @@ -98,12 +98,12 @@
"shellcheck": "0.10.0",
"shfmt": "3.9.0",
"snakefmt": "0.10.2",
"snakemake": "8.20.6",
"snakemake": "8.20.7",
"spectral": "6.13.1",
"sql-lint": "1.0.0",
"sqlfluff": "3.2.2",
"sqlfluff": "3.2.3",
"standard": "17.1.2",
"stylelint": "16.9.0",
"stylelint": "16.10.0",
"stylua": "0.20.0",
"swiftlint": "0.57.0",
"syft": "1.14.0",
Expand All @@ -112,8 +112,8 @@
"terragrunt": "0.67.5",
"terrascan": "1.19.9",
"tflint": "0.53.0",
"trivy": "0.56.1",
"trivy-sbom": "0.56.1",
"trivy": "0.56.2",
"trivy-sbom": "0.56.2",
"trufflehog": "3.82.7",
"ts-standard": "12.0.2",
"tsqllint": "1.15.3.0",
Expand Down
14 changes: 14 additions & 0 deletions .github/workflows/deploy-ALPHA-flavors.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -134,6 +134,20 @@ jobs:
tags: |
ghcr.io/oxsecurity/megalinter-worker-${{ matrix.flavor }}:alpha

- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}

# Copy ghrc.io image to Docker Hub
- name: Pull image from GHCR
run: docker pull ghcr.io/oxsecurity/megalinter-worker-${{ matrix.flavor }}:alpha
- name: Tag image for Docker Hub
run: docker tag ghcr.io/oxsecurity/megalinter-worker-${{ matrix.flavor }}:alpha oxsecurity/megalinter-worker-${{ matrix.flavor }}:alpha
- name: Push image to Docker Hub
run: docker push oxsecurity/megalinter-worker-${{ matrix.flavor }}:alpha

##############################################
# Check Docker image security with Trivy #
##############################################
Expand Down
13 changes: 13 additions & 0 deletions .github/workflows/deploy-ALPHA.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -142,3 +142,16 @@ jobs:
tags: |
ghcr.io/oxsecurity/megalinter-worker:alpha

- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}

# Copy ghrc.io image to Docker Hub
- name: Pull image from GHCR
run: docker pull ghcr.io/oxsecurity/megalinter:alpha
- name: Tag image for Docker Hub
run: docker tag ghcr.io/oxsecurity/megalinter:alpha oxsecurity/megalinter:alpha
- name: Push image to Docker Hub
run: docker push oxsecurity/megalinter:alpha
65 changes: 56 additions & 9 deletions .github/workflows/deploy-BETA-flavors.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,39 +94,56 @@ jobs:
id: meta
with:
images: |
${{ github.repository }}-${{ matrix.flavor }}
ghcr.io/${{ github.repository }}-${{ matrix.flavor }}
flavor: |
latest=false
prefix=beta
tags: |
type=raw,value=

- name: Docker Metadata action (Docker Hub)
uses: docker/[email protected]
id: meta-dhub
with:
images: |
${{ github.repository }}-${{ matrix.flavor }}
flavor: |
latest=false
prefix=beta
tags: |
type=raw,value=

- name: Docker Metadata action (Worker)
uses: docker/[email protected]
id: meta-w
with:
images: |
name=${{ github.repository }}-worker-${{ matrix.flavor }},enable=false
name=ghcr.io/${{ github.repository }}-worker-${{ matrix.flavor }}
flavor: |
latest=false
prefix=beta
tags: |
type=raw,value=

- name: Docker Metadata action (Worker Docker Hub)
uses: docker/[email protected]
id: meta-w-dhub
with:
images: |
name=${{ github.repository }}-worker-${{ matrix.flavor }},enable=false
flavor: |
latest=false
prefix=beta
tags: |
type=raw,value=

- name: Set up QEMU
uses: docker/setup-qemu-action@v3
if: ${{ ( ( runner.arch != 'X64' || runner.os != 'Linux' ) && matrix.platform == 'linux/amd64' ) || matrix.platform != 'linux/amd64' }}

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}

- name: Login to GitHub Container Registry
uses: docker/login-action@v3
with:
Expand Down Expand Up @@ -172,7 +189,37 @@ jobs:
push: ${{ github.event_name != 'pull_request' }}
secrets: |
GITHUB_TOKEN=${{ secrets.GITHUB_TOKEN }}
tags: ${{ steps.meta.outputs.tags }}
tags: ${{ steps.meta-w.outputs.tags }}

- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}

# Copy ghrc.io image to Docker Hub (main image)
- name: Pull image from GHCR
run: docker pull ${{ steps.meta.outputs.tags[0] }}
- name: Tag image for Docker Hub
run: docker tag ${{ steps.meta.outputs.tags[0] }} ${{ steps.meta-dhub.outputs.tags[0] }}
- name: Push image to Docker Hub
run: docker push ${{ steps.meta-dhub.outputs.tags[0] }}

# Copy ghrc.io image to Docker Hub (server)
- name: Pull image from GHCR
run: docker pull ${{ steps.meta-s.outputs.tags[0] }}
- name: Tag image for Docker Hub
run: docker tag ${{ steps.meta-s.outputs.tags[0] }} ${{ steps.meta-s-dhub.outputs.tags[0] }}
- name: Push image to Docker Hub
run: docker push ${{ steps.meta-s-dhub.outputs.tags[0] }}

# Copy ghrc.io image to Docker Hub (worker)
- name: Pull image from GHCR
run: docker pull ${{ steps.meta-w.outputs.tags[0] }}
- name: Tag image for Docker Hub
run: docker tag ${{ steps.meta-w.outputs.tags[0] }} ${{ steps.meta-w-dhub.outputs.tags[0] }}
- name: Push image to Docker Hub
run: docker push ${{ steps.meta-w-dhub.outputs.tags[0] }}

##############################################
# Check Docker image security with Trivy #
Expand Down
34 changes: 27 additions & 7 deletions .github/workflows/deploy-BETA-linters.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -195,7 +195,6 @@ jobs:
id: meta
with:
images: |
${{ github.repository }}-only-${{ matrix.linter }}
ghcr.io/${{ github.repository }}-only-${{ matrix.linter }}
flavor: |
latest=false
Expand All @@ -204,19 +203,26 @@ jobs:
type=raw,value=
type=raw,value={{date 'YYYYMMDD_HHmm'}}

- name: Docker Metadata action (Docker hub)
uses: docker/[email protected]
id: meta-dhub
with:
images: |
${{ github.repository }}-only-${{ matrix.linter }}
flavor: |
latest=false
prefix=beta
tags: |
type=raw,value=
type=raw,value={{date 'YYYYMMDD_HHmm'}}

- name: Set up QEMU
uses: docker/setup-qemu-action@v3
if: ${{ ( ( runner.arch != 'X64' || runner.os != 'Linux' ) && matrix.platform == 'linux/amd64' ) || matrix.platform != 'linux/amd64' }}

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}

- name: Login to GitHub Container Registry
uses: docker/login-action@v3
with:
Expand Down Expand Up @@ -255,6 +261,20 @@ jobs:
docker run -e TEST_CASE_RUN=true -e OUTPUT_FORMAT=text -e OUTPUT_FOLDER=${{ github.sha }} -e OUTPUT_DETAIL=detailed -e GITHUB_SHA=${{ github.sha }} -e GITHUB_REPOSITORY=${GITHUB_REPOSITORY} -e GITHUB_BRANCH=${GITHUB_BRANCH} -e GITHUB_TOKEN="${{ secrets.GITHUB_TOKEN }}" -e TEST_KEYWORDS="${TEST_KEYWORDS_TO_USE}" -e MEGALINTER_VOLUME_ROOT="${GITHUB_WORKSPACE}" -v "/var/run/docker.sock:/var/run/docker.sock:rw" -v ${GITHUB_WORKSPACE}:/tmp/lint ${{ fromJson(steps.meta.outputs.json).tags[0]}}
timeout-minutes: 30

- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}

# Copy ghrc.io image to Docker Hub (main image)
- name: Pull image from GHCR
run: docker pull ${{ steps.meta.outputs.tags[0] }}
- name: Tag image for Docker Hub
run: docker tag ${{ steps.meta.outputs.tags[0] }} ${{ steps.meta-dhub.outputs.tags[0] }}
- name: Push image to Docker Hub
run: docker push ${{ steps.meta-dhub.outputs.tags[0] }}

##############################################
# Check Docker image security with Trivy #
##############################################
Expand Down
66 changes: 57 additions & 9 deletions .github/workflows/deploy-BETA.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,43 +83,61 @@ jobs:
id: meta
with:
images: |
name=${{ github.repository }}
name=ghcr.io/${{ github.repository }}
tags: |
type=raw,value=beta

- name: Docker Metadata action (Docker Hub)
uses: docker/[email protected]
id: meta-dhub
with:
images: |
name=${{ github.repository }}
tags: |
type=raw,value=beta

- name: Docker Metadata action (Server)
uses: docker/[email protected]
id: meta-s
with:
images: |
name=${{ github.repository }}-server,enable=false
name=ghcr.io/${{ github.repository }}-server
tags: |
type=raw,value=beta

- name: Docker Metadata action (Server Docker Hub)
uses: docker/[email protected]
id: meta-s-dhub
with:
images: |
name=${{ github.repository }}-server,enable=false
tags: |
type=raw,value=beta

- name: Docker Metadata action (Worker)
uses: docker/[email protected]
id: meta-w
with:
images: |
name=${{ github.repository }}-worker,enable=false
name=ghcr.io/${{ github.repository }}-worker
tags: |
type=raw,value=beta

- name: Docker Metadata action (Worker Server)
uses: docker/[email protected]
id: meta-w-dhub
with:
images: |
name=${{ github.repository }}-worker,enable=false
tags: |
type=raw,value=beta

- name: Set up QEMU
uses: docker/setup-qemu-action@v3

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}

- name: Login to GitHub Container Registry
uses: docker/login-action@v3
with:
Expand Down Expand Up @@ -176,6 +194,36 @@ jobs:
GITHUB_TOKEN=${{ secrets.GITHUB_TOKEN }}
tags: ${{ steps.meta-w.outputs.tags }}

- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}

# Copy ghrc.io image to Docker Hub (main image)
- name: Pull image from GHCR
run: docker pull ${{ steps.meta.outputs.tags[0] }}
- name: Tag image for Docker Hub
run: docker tag ${{ steps.meta.outputs.tags[0] }} ${{ steps.meta-dhub.outputs.tags[0] }}
- name: Push image to Docker Hub
run: docker push ${{ steps.meta-dhub.outputs.tags[0] }}

# Copy ghrc.io image to Docker Hub (server)
- name: Pull image from GHCR
run: docker pull ${{ steps.meta-s.outputs.tags[0] }}
- name: Tag image for Docker Hub
run: docker tag ${{ steps.meta-s.outputs.tags[0] }} ${{ steps.meta-s-dhub.outputs.tags[0] }}
- name: Push image to Docker Hub
run: docker push ${{ steps.meta-s-dhub.outputs.tags[0] }}

# Copy ghrc.io image to Docker Hub (worker)
- name: Pull image from GHCR
run: docker pull ${{ steps.meta-w.outputs.tags[0] }}
- name: Tag image for Docker Hub
run: docker tag ${{ steps.meta-w.outputs.tags[0] }} ${{ steps.meta-w-dhub.outputs.tags[0] }}
- name: Push image to Docker Hub
run: docker push ${{ steps.meta-w-dhub.outputs.tags[0] }}

# ###############################
# # Run tests for code coverage #
# ###############################
Expand Down
3 changes: 3 additions & 0 deletions .trivyignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,9 @@ CVE-2024-0057
CVE-2024-30105
CVE-2024-38095
CVE-2024-38168
CVE-2024-43483
CVE-2024-43484
CVE-2024-43485

# Docker
CVE-2024-41110
Expand Down
Loading
Loading