Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SYCL][Devops] Fix DockerFile linting issues discovered by trivy #16361

Closed
Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
045e856
[SYCL][Devops] Fix AVD-DS-0017
AlexeySachkov Dec 5, 2024
15a7c6a
[SYCL][Devops] Fix AVD-DS-0002
AlexeySachkov Dec 5, 2024
10cc0e3
[SYCL][Devops] Fix remaining AVD-DS-0002 issues
AlexeySachkov Dec 5, 2024
1ea04f5
An attempt to fix docker images build
AlexeySachkov Dec 6, 2024
49ab319
Outline sycl user creation into a separate script; Make it require pa…
AlexeySachkov Dec 13, 2024
c0439ff
Drop seemingly dead legacy code
AlexeySachkov Dec 13, 2024
35a07f0
Propagate new secret to images and use sudo with password in workflows
AlexeySachkov Dec 13, 2024
ed7171c
Run various driver installs as root when building containers
AlexeySachkov Dec 13, 2024
1ff5dd4
Properly propagate secrets down to user creation script
AlexeySachkov Dec 13, 2024
4d28c77
Properly use new secret in build-container action
AlexeySachkov Dec 13, 2024
8530141
Use the right secret
AlexeySachkov Dec 13, 2024
3d7c76f
Access action inputs properly
AlexeySachkov Dec 13, 2024
3cb941a
Permissions & action fixes
AlexeySachkov Dec 13, 2024
6d88ec7
Lowercase root
AlexeySachkov Dec 16, 2024
0964ce8
Differentiate password-less sycl vs sycl-ci user
AlexeySachkov Dec 16, 2024
0625177
Align secret name with #16384
AlexeySachkov Dec 16, 2024
be4e0c6
Merge remote-tracking branch 'origin/sycl' into private/asachkov/appl…
AlexeySachkov Dec 18, 2024
ff333a1
Fix trivy reported issues in a new docker file
AlexeySachkov Dec 18, 2024
8d931df
Switch all containers to `sycl_ci` user and update the documentation
AlexeySachkov Dec 18, 2024
5d82571
Fix a typo
AlexeySachkov Dec 18, 2024
4ef8161
One more typo fix
AlexeySachkov Dec 18, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/sycl-containers.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@ jobs:
file: ${{ matrix.file }}
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}
sycl_passwd: ${{ secrets.DOCKER_SUDO_PASSWORD }}
tags: |
ghcr.io/${{ github.repository }}/${{ matrix.file }}:${{ matrix.tag }}-${{ github.sha }}
ghcr.io/${{ github.repository }}/${{ matrix.file }}:${{ matrix.tag }}
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/sycl-linux-run-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -159,8 +159,8 @@ jobs:
- name: Reset Intel GPU
if: inputs.reset_intel_gpu == 'true'
run: |
sudo mount -t debugfs none /sys/kernel/debug
sudo bash -c 'echo 1 > /sys/kernel/debug/dri/0/i915_wedged'
cat /run/secrets/sycl_passwd | sudo -S mount -t debugfs none /sys/kernel/debug
cat /run/secrets/sycl_passwd | sudo -S bash -c 'echo 1 > /sys/kernel/debug/dri/0/i915_wedged'
- uses: actions/checkout@v4
with:
ref: ${{ inputs.ref }}
Expand Down Expand Up @@ -196,9 +196,9 @@ jobs:
run: |
if [ "${{ inputs.install_dev_igc_driver }}" = "true" ]; then
# If libllvm14 is already installed (dev igc docker), still return true.
sudo apt-get install -yqq libllvm14 || true;
cat /run/secrets/sycl_passwd | sudo -S apt-get install -yqq libllvm14 || true;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need parens here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From my local experiments:

$ cat /run/secrets/sycl_ci_passwd | sudo -S echo "I'm root!" || echo "second"
I'm root!
$ cat /run/secrets/sycl_ci_passwd | sudo -S false || echo "second"
second

So, it seems to work correctly, but I can add parens for clarity if that's a preference

fi
sudo -E bash devops/scripts/install_drivers.sh llvm/devops/dependencies.json ${{ inputs.install_dev_igc_driver == 'true' && 'llvm/devops/dependencies-igc-dev.json --use-dev-igc' || '' }} --all
cat /run/secrets/sycl_passwd | sudo -S -E bash devops/scripts/install_drivers.sh llvm/devops/dependencies.json ${{ inputs.install_dev_igc_driver == 'true' && 'llvm/devops/dependencies-igc-dev.json --use-dev-igc' || '' }} --all
- name: Source OneAPI TBB vars.sh
shell: bash
run: |
Expand Down
5 changes: 5 additions & 0 deletions devops/actions/build_container/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,9 @@ inputs:
file:
description: "Dockerfile"
required: true
sycl_passwd:
AlexeySachkov marked this conversation as resolved.
Show resolved Hide resolved
description: "Password to assign to sycl user within a container"
required: true

runs:
using: "composite"
Expand All @@ -31,6 +34,7 @@ runs:
registry: ghcr.io
username: ${{ inputs.username }}
password: ${{ inputs.password }}
sycl_passwd: ${{ inputs.sycl_passwd }}
- name: Build and Push Container
uses: docker/[email protected]
with:
Expand All @@ -41,3 +45,4 @@ runs:
file: ${{ github.workspace }}/devops/containers/${{ inputs.file }}.Dockerfile
secrets: |
github_token=${{ github.token }}
sycl_passwd=${{ inputs.sycl_passwd }}
19 changes: 4 additions & 15 deletions devops/containers/ubuntu2204_base.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -8,25 +8,14 @@ USER root
COPY scripts/install_build_tools.sh /install.sh
RUN /install.sh

# By default Ubuntu sets an arbitrary UID value, that is different from host
# system. When CI passes default UID value of 1001, some of LLVM tools fail to
# discover user home directory and fail a few LIT tests. Fixes UID and GID to
# 1001, that is used as default by GitHub Actions.
RUN groupadd -g 1001 sycl && useradd sycl -u 1001 -g 1001 -m -s /bin/bash
# Add sycl user to video/irc groups so that it can access GPU
RUN usermod -aG video sycl
RUN usermod -aG irc sycl

# group 109 is required for sycl user to access PVC card.
RUN groupadd -g 109 render
RUN usermod -aG render sycl

# Allow sycl user to run as sudo
RUN echo "sycl ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers
COPY scripts/create-sycl-user.sh /user-setup.sh
RUN --mount=type=secret,id=sycl_passwd /user-setup.sh

COPY actions/cached_checkout /actions/cached_checkout
COPY actions/cleanup /actions/cleanup
COPY scripts/docker_entrypoint.sh /docker_entrypoint.sh
COPY scripts/install_drivers.sh /opt/install_drivers.sh

USER sycl

ENTRYPOINT ["/docker_entrypoint.sh"]
17 changes: 6 additions & 11 deletions devops/containers/ubuntu2204_build.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -24,23 +24,18 @@ gpg --dearmor | tee /etc/apt/keyrings/rocm.gpg > /dev/null && \
# Add rocm repo
echo "deb [arch=amd64 signed-by=/etc/apt/keyrings/rocm.gpg] https://repo.radeon.com/rocm/apt/6.1.1 jammy main" \
| tee --append /etc/apt/sources.list.d/rocm.list && \
printf 'Package: *\nPin: release o=repo.radeon.com\nPin-Priority: 600' | tee /etc/apt/preferences.d/rocm-pin-600 && \
apt update
printf 'Package: *\nPin: release o=repo.radeon.com\nPin-Priority: 600' | tee /etc/apt/preferences.d/rocm-pin-600
# Install the kernel driver
RUN apt install -yqq rocm-dev && \
RUN apt update && apt install -yqq rocm-dev && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*

# By default Ubuntu sets an arbitrary UID value, that is different from host
# system. When CI passes default UID value of 1001, some of LLVM tools fail to
# discover user home directory and fail a few LIT tests. Fixes UID and GID to
# 1001, that is used as default by GitHub Actions.
RUN groupadd -g 1001 sycl && useradd sycl -u 1001 -g 1001 -m -s /bin/bash
# Add sycl user to video/irc groups so that it can access GPU
RUN usermod -aG video sycl
RUN usermod -aG irc sycl
COPY scripts/create-sycl-user.sh /user-setup.sh
RUN --mount=type=secret,id=sycl_passwd /user-setup.sh

COPY scripts/docker_entrypoint.sh /docker_entrypoint.sh

USER sycl

ENTRYPOINT ["/docker_entrypoint.sh"]

4 changes: 4 additions & 0 deletions devops/containers/ubuntu2204_intel_drivers.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ ENV DEBIAN_FRONTEND=noninteractive

ARG use_latest=true

USER root

RUN apt update && apt install -yqq wget

COPY scripts/get_release.py /
Expand All @@ -25,5 +27,7 @@ RUN --mount=type=secret,id=github_token \

COPY scripts/drivers_entrypoint.sh /drivers_entrypoint.sh

USER sycl

ENTRYPOINT ["/bin/bash", "/drivers_entrypoint.sh"]

4 changes: 4 additions & 0 deletions devops/containers/ubuntu2204_preinstalled.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,16 @@ ARG base_image=ghcr.io/intel/llvm/ubuntu2204_intel_drivers

FROM $base_image:$base_tag

USER ROOT
AlexeySachkov marked this conversation as resolved.
Show resolved Hide resolved

COPY scripts/drivers_entrypoint.sh /drivers_entrypoint.sh
RUN mkdir -p /opt/sycl
ADD sycl_linux.tar.gz /opt/sycl/

ENV PATH /opt/sycl/bin:$PATH
ENV LD_LIBRARY_PATH /opt/sycl/lib:$LD_LIBRARY_PATH

USER sycl

ENTRYPOINT ["/bin/bash", "/drivers_entrypoint.sh"]

19 changes: 4 additions & 15 deletions devops/containers/ubuntu2404_base.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -8,25 +8,14 @@ USER root
COPY scripts/install_build_tools.sh /install.sh
RUN /install.sh

# By default Ubuntu sets an arbitrary UID value, that is different from host
# system. When CI passes default UID value of 1001, some of LLVM tools fail to
# discover user home directory and fail a few LIT tests. Fixes UID and GID to
# 1001, that is used as default by GitHub Actions.
RUN groupadd -g 1001 sycl && useradd sycl -u 1001 -g 1001 -m -s /bin/bash
# Add sycl user to video/irc groups so that it can access GPU
RUN usermod -aG video sycl
RUN usermod -aG irc sycl

# group 109 is required for sycl user to access PVC card.
RUN groupadd -g 109 render
RUN usermod -aG render sycl

# Allow sycl user to run as sudo
RUN echo "sycl ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers
COPY scripts/create-sycl-user.sh /user-setup.sh
RUN --mount=type=secret,id=sycl_passwd /user-setup.sh

COPY actions/cached_checkout /actions/cached_checkout
COPY actions/cleanup /actions/cleanup
COPY scripts/docker_entrypoint.sh /docker_entrypoint.sh
COPY scripts/install_drivers.sh /opt/install_drivers.sh

USER sycl

ENTRYPOINT ["/docker_entrypoint.sh"]
4 changes: 4 additions & 0 deletions devops/containers/ubuntu2404_intel_drivers.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ ENV DEBIAN_FRONTEND=noninteractive

ARG use_latest=true

USER root

RUN apt update && apt install -yqq wget

COPY scripts/get_release.py /
Expand All @@ -25,5 +27,7 @@ RUN --mount=type=secret,id=github_token \

COPY scripts/drivers_entrypoint.sh /drivers_entrypoint.sh

USER sycl

ENTRYPOINT ["/bin/bash", "/drivers_entrypoint.sh"]

Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ FROM $base_image:$base_tag

ENV DEBIAN_FRONTEND=noninteractive

USER root

RUN apt update && apt install -yqq libllvm14

COPY scripts/get_release.py /
Expand All @@ -20,5 +22,7 @@ RUN --mount=type=secret,id=github_token \

COPY scripts/drivers_entrypoint.sh /drivers_entrypoint.sh

USER sycl

ENTRYPOINT ["/bin/bash", "/drivers_entrypoint.sh"]

30 changes: 30 additions & 0 deletions devops/scripts/create-sycl-user.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
#!/bin/bash

# By default Ubuntu sets an arbitrary UID value, that is different from host
# system. When CI passes default UID value of 1001, some of LLVM tools fail to
# discover user home directory and fail a few LIT tests. Fixes UID and GID to
# 1001, that is used as default by GitHub Actions.
groupadd -g 1001 sycl && useradd sycl -u 1001 -g 1001 -m -s /bin/bash
# Add sycl user to video/irc groups so that it can access GPU
usermod -aG video sycl
usermod -aG irc sycl

# group 109 is required for sycl user to access PVC card.
groupadd -g 109 render
usermod -aG render sycl

if [[ -f /run/secrets/sycl_passwd ]]; then
# When running in our CI environment, we restrict access to root.

# Set password for sycl user
cat /run/secrets/sycl_passwd | passwd -s sycl

# Allow sycl user to run as sudo, but only with password
echo "sycl ALL=(root) PASSWD:ALL" >> /etc/sudoers
else
# Otherwise, we allow password-less root to simplify building other
AlexeySachkov marked this conversation as resolved.
Show resolved Hide resolved
# containers on top.

# Allow sycl user to run as sudo passwrod-less
echo "sycl ALL=(root) NOPASSWD:ALL" >> /etc/sudoers
fi
5 changes: 0 additions & 5 deletions devops/scripts/docker_entrypoint.sh
Original file line number Diff line number Diff line change
@@ -1,8 +1,3 @@
#!/bin/bash

if [ -d "$GITHUB_WORKSPACE" ]; then
chown -R sycl:sycl $GITHUB_WORKSPACE
su sycl
fi

exec "$@"
Loading