Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SYCL][Devops] Fix DockerFile linting issues discovered by trivy #16411

Merged
merged 15 commits into from
Dec 19, 2024
Merged
Show file tree
Hide file tree
Changes from 13 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions .github/workflows/sycl-linux-run-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -159,8 +159,8 @@ jobs:
- name: Reset Intel GPU
if: inputs.reset_intel_gpu == 'true'
run: |
sudo mount -t debugfs none /sys/kernel/debug
sudo bash -c 'echo 1 > /sys/kernel/debug/dri/0/i915_wedged'
cat /run/secrets/sycl_passwd | sudo -S mount -t debugfs none /sys/kernel/debug
cat /run/secrets/sycl_passwd | sudo -S bash -c 'echo 1 > /sys/kernel/debug/dri/0/i915_wedged'
- uses: actions/checkout@v4
with:
ref: ${{ inputs.ref }}
Expand Down Expand Up @@ -196,9 +196,9 @@ jobs:
run: |
if [ "${{ inputs.install_dev_igc_driver }}" = "true" ]; then
# If libllvm14 is already installed (dev igc docker), still return true.
sudo apt-get install -yqq libllvm14 || true;
cat /run/secrets/sycl_passwd | sudo -S apt-get install -yqq libllvm14 || true;
fi
sudo -E bash devops/scripts/install_drivers.sh llvm/devops/dependencies.json ${{ inputs.install_dev_igc_driver == 'true' && 'llvm/devops/dependencies-igc-dev.json --use-dev-igc' || '' }} --all
cat /run/secrets/sycl_passwd | sudo -S -E bash devops/scripts/install_drivers.sh llvm/devops/dependencies.json ${{ inputs.install_dev_igc_driver == 'true' && 'llvm/devops/dependencies-igc-dev.json --use-dev-igc' || '' }} --all
- name: Source OneAPI TBB vars.sh
shell: bash
run: |
Expand Down
19 changes: 4 additions & 15 deletions devops/containers/ubuntu2204_base.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -8,25 +8,14 @@ USER root
COPY scripts/install_build_tools.sh /install.sh
RUN /install.sh

# By default Ubuntu sets an arbitrary UID value, that is different from host
# system. When CI passes default UID value of 1001, some of LLVM tools fail to
# discover user home directory and fail a few LIT tests. Fixes UID and GID to
# 1001, that is used as default by GitHub Actions.
RUN groupadd -g 1001 sycl && useradd sycl -u 1001 -g 1001 -m -s /bin/bash
# Add sycl user to video/irc groups so that it can access GPU
RUN usermod -aG video sycl
RUN usermod -aG irc sycl

# group 109 is required for sycl user to access PVC card.
RUN groupadd -g 109 render
RUN usermod -aG render sycl

# Allow sycl user to run as sudo
RUN echo "sycl ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers
COPY scripts/create-sycl-user.sh /user-setup.sh
RUN --mount=type=secret,id=sycl_ci_passwd /user-setup.sh

COPY actions/cached_checkout /actions/cached_checkout
COPY actions/cleanup /actions/cleanup
COPY scripts/docker_entrypoint.sh /docker_entrypoint.sh
COPY scripts/install_drivers.sh /opt/install_drivers.sh

USER sycl_ci

ENTRYPOINT ["/docker_entrypoint.sh"]
17 changes: 6 additions & 11 deletions devops/containers/ubuntu2204_build.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -24,23 +24,18 @@ gpg --dearmor | tee /etc/apt/keyrings/rocm.gpg > /dev/null && \
# Add rocm repo
echo "deb [arch=amd64 signed-by=/etc/apt/keyrings/rocm.gpg] https://repo.radeon.com/rocm/apt/6.1.1 jammy main" \
| tee --append /etc/apt/sources.list.d/rocm.list && \
printf 'Package: *\nPin: release o=repo.radeon.com\nPin-Priority: 600' | tee /etc/apt/preferences.d/rocm-pin-600 && \
apt update
printf 'Package: *\nPin: release o=repo.radeon.com\nPin-Priority: 600' | tee /etc/apt/preferences.d/rocm-pin-600
# Install the kernel driver
RUN apt install -yqq rocm-dev && \
RUN apt update && apt install -yqq rocm-dev && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*

# By default Ubuntu sets an arbitrary UID value, that is different from host
# system. When CI passes default UID value of 1001, some of LLVM tools fail to
# discover user home directory and fail a few LIT tests. Fixes UID and GID to
# 1001, that is used as default by GitHub Actions.
RUN groupadd -g 1001 sycl && useradd sycl -u 1001 -g 1001 -m -s /bin/bash
# Add sycl user to video/irc groups so that it can access GPU
RUN usermod -aG video sycl
RUN usermod -aG irc sycl
COPY scripts/create-sycl-user.sh /user-setup.sh
RUN --mount=type=secret,id=sycl_ci_passwd /user-setup.sh

COPY scripts/docker_entrypoint.sh /docker_entrypoint.sh

USER sycl_ci

ENTRYPOINT ["/docker_entrypoint.sh"]

4 changes: 4 additions & 0 deletions devops/containers/ubuntu2204_intel_drivers.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ ENV DEBIAN_FRONTEND=noninteractive

ARG use_latest=true

USER root

RUN apt update && apt install -yqq wget

COPY scripts/get_release.py /
Expand All @@ -25,5 +27,7 @@ RUN --mount=type=secret,id=github_token \

COPY scripts/drivers_entrypoint.sh /drivers_entrypoint.sh

USER sycl_ci

ENTRYPOINT ["/bin/bash", "/drivers_entrypoint.sh"]

4 changes: 4 additions & 0 deletions devops/containers/ubuntu2204_preinstalled.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,16 @@ ARG base_image=ghcr.io/intel/llvm/ubuntu2204_intel_drivers

FROM $base_image:$base_tag

USER root

COPY scripts/drivers_entrypoint.sh /drivers_entrypoint.sh
RUN mkdir -p /opt/sycl
ADD sycl_linux.tar.gz /opt/sycl/

ENV PATH /opt/sycl/bin:$PATH
ENV LD_LIBRARY_PATH /opt/sycl/lib:$LD_LIBRARY_PATH

USER sycl_ci

ENTRYPOINT ["/bin/bash", "/drivers_entrypoint.sh"]

19 changes: 4 additions & 15 deletions devops/containers/ubuntu2404_base.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -8,25 +8,14 @@ USER root
COPY scripts/install_build_tools.sh /install.sh
RUN /install.sh

# By default Ubuntu sets an arbitrary UID value, that is different from host
# system. When CI passes default UID value of 1001, some of LLVM tools fail to
# discover user home directory and fail a few LIT tests. Fixes UID and GID to
# 1001, that is used as default by GitHub Actions.
RUN groupadd -g 1001 sycl && useradd sycl -u 1001 -g 1001 -m -s /bin/bash
# Add sycl user to video/irc groups so that it can access GPU
RUN usermod -aG video sycl
RUN usermod -aG irc sycl

# group 109 is required for sycl user to access PVC card.
RUN groupadd -g 109 render
RUN usermod -aG render sycl

# Allow sycl user to run as sudo
RUN echo "sycl ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers
COPY scripts/create-sycl-user.sh /user-setup.sh
RUN --mount=type=secret,id=sycl_ci_passwd /user-setup.sh

COPY actions/cached_checkout /actions/cached_checkout
COPY actions/cleanup /actions/cleanup
COPY scripts/docker_entrypoint.sh /docker_entrypoint.sh
COPY scripts/install_drivers.sh /opt/install_drivers.sh

USER sycl_ci

ENTRYPOINT ["/docker_entrypoint.sh"]
18 changes: 7 additions & 11 deletions devops/containers/ubuntu2404_build_oneapi.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -31,23 +31,19 @@ echo -e 'Package: *\nPin: release o=repo.radeon.com\nPin-Priority: 600' \
wget -O- https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB | gpg --dearmor \
| tee /usr/share/keyrings/oneapi-archive-keyring.gpg > /dev/null && \
echo "deb [signed-by=/usr/share/keyrings/oneapi-archive-keyring.gpg] https://apt.repos.intel.com/oneapi all main" \
| tee /etc/apt/sources.list.d/oneAPI.list && \
apt update
| tee /etc/apt/sources.list.d/oneAPI.list

# Install the ROCM kernel driver and oneAPI
RUN apt install -yqq rocm-dev intel-oneapi-compiler-dpcpp-cpp && \
RUN apt update && apt install -yqq rocm-dev intel-oneapi-compiler-dpcpp-cpp && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*

# By default Ubuntu sets an arbitrary UID value, that is different from host
# system. When CI passes default UID value of 1001, some of LLVM tools fail to
# discover user home directory and fail a few LIT tests. Fixes UID and GID to
# 1001, that is used as default by GitHub Actions.
RUN groupadd -g 1001 sycl && useradd sycl -u 1001 -g 1001 -m -s /bin/bash
# Add sycl user to video/irc groups so that it can access GPU
RUN usermod -aG video sycl
RUN usermod -aG irc sycl
COPY scripts/create-sycl-user.sh /user-setup.sh
RUN --mount=type=secret,id=sycl_ci_passwd /user-setup.sh

COPY scripts/docker_entrypoint.sh /docker_entrypoint.sh

USER sycl_ci

ENTRYPOINT ["/docker_entrypoint.sh"]

4 changes: 4 additions & 0 deletions devops/containers/ubuntu2404_intel_drivers.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ ENV DEBIAN_FRONTEND=noninteractive

ARG use_latest=true

USER root

RUN apt update && apt install -yqq wget

COPY scripts/get_release.py /
Expand All @@ -25,5 +27,7 @@ RUN --mount=type=secret,id=github_token \

COPY scripts/drivers_entrypoint.sh /drivers_entrypoint.sh

USER sycl_ci

ENTRYPOINT ["/bin/bash", "/drivers_entrypoint.sh"]

Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ FROM $base_image:$base_tag

ENV DEBIAN_FRONTEND=noninteractive

USER root

RUN apt update && apt install -yqq libllvm14

COPY scripts/get_release.py /
Expand All @@ -20,5 +22,7 @@ RUN --mount=type=secret,id=github_token \

COPY scripts/drivers_entrypoint.sh /drivers_entrypoint.sh

USER sycl_ci

ENTRYPOINT ["/bin/bash", "/drivers_entrypoint.sh"]

50 changes: 50 additions & 0 deletions devops/scripts/create-sycl-user.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
#!/bin/bash

set -e

if [[ $# -eq 0 ]]; then
# When launched without arguments, we assume that it was launched as part of
# CI workflow and therefore a different kind of user is created
USER_NAME=sycl_ci
SET_PASSWD=true

# By default Ubuntu sets an arbitrary UID value, that is different from host
# system. When CI passes default UID value of 1001, some of LLVM tools fail to
# discover user home directory and fail a few LIT tests. Fixes UID and GID to
# 1001, that is used as default by GitHub Actions.
USER_ID=1001
else
if [[ "${1:-}" != "--regular" ]]; then
echo "The only supported argument is --regular!"
exit 1
fi
USER_NAME=sycl
SET_PASSWD=false

# Some user id which is different from the one assigned to sycl_ci user
USER_ID=1234
fi

groupadd -g $USER_ID $USER_NAME && useradd $USER_NAME -u $USER_ID -g $USER_ID -m -s /bin/bash
# Add user to video/irc groups so that it can access GPU
usermod -aG video $USER_NAME
usermod -aG irc $USER_NAME

# group 109 is required for user to access PVC card.
groupadd -f -g 109 render
usermod -aG render $USER_NAME

if [[ $SET_PASSWD == true ]]; then
if [[ ! -f /run/secrets/sycl_ci_passwd ]]; then
uditagarwal97 marked this conversation as resolved.
Show resolved Hide resolved
echo "Password is requested, but /run/secrets/sycl_ci_passwd doesn't exist!"
exit 2
fi

# Set password for user
echo "$USER_NAME:$(cat /run/secrets/sycl_ci_passwd)" | chpasswd

# Allow user to run as sudo, but only with password
echo "$USER_NAME ALL=(ALL) PASSWD:ALL" >> /etc/sudoers
else
echo "$USER_NAME ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers
fi
5 changes: 0 additions & 5 deletions devops/scripts/docker_entrypoint.sh
Original file line number Diff line number Diff line change
@@ -1,8 +1,3 @@
#!/bin/bash

if [ -d "$GITHUB_WORKSPACE" ]; then
chown -R sycl:sycl $GITHUB_WORKSPACE
su sycl
fi

exec "$@"
18 changes: 14 additions & 4 deletions sycl/doc/developer/DockerBKMs.md
Original file line number Diff line number Diff line change
Expand Up @@ -143,10 +143,20 @@ instructions.

## Changing Docker user

By default all processes inside Docker run as root. Some LLVM or Clang tests
expect your user to be anything but root. You can change the user by specifying
`-u <username or uid>` option. All Docker containers come with user `sycl`
created.
By default all processes within our containers are run as `sycl_ci` user.
AlexeySachkov marked this conversation as resolved.
Show resolved Hide resolved
Note: it **does not** have password-less `root` access.

If you want to change the user, you can do that by specifying
AlexeySachkov marked this conversation as resolved.
Show resolved Hide resolved
`-u <username or uid>` option when running the container.

All containers come with `/user-setup.sh` script which can used to create `sycl`
AlexeySachkov marked this conversation as resolved.
Show resolved Hide resolved
user which has all the same groups as `sycl_ci` user, but also has passwrod-less
AlexeySachkov marked this conversation as resolved.
Show resolved Hide resolved
access to `root`. Use the script as follows:

```bash
# Note: the script requires root permissions to create a new user
/user-setup.sh --regular
AlexeySachkov marked this conversation as resolved.
Show resolved Hide resolved
```
bader marked this conversation as resolved.
Show resolved Hide resolved

## Managing downloaded Docker images

Expand Down
Loading