Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for WSL2 and Docker Desktop for Windows #80

Open
wants to merge 3 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@
sles_version_numbers = ['15.5', '15.4']
ol_release_version_numbers = ['8']
ol_version_numbers = [('8', '8.8')]
ubuntu_wsl2_docker_desktop_win_version_numbers = [('22.04', 'jammy')]

# pages with specific settings
article_pages = [
Expand Down Expand Up @@ -85,5 +86,6 @@
"rhel_release_version_numbers" : rhel_release_version_numbers,
"rhel_version_numbers" : rhel_version_numbers,
"ol_release_version_numbers" : ol_release_version_numbers,
"ol_version_numbers" : ol_version_numbers
"ol_version_numbers" : ol_version_numbers,
"ubuntu_wsl2_docker_desktop_win_version_numbers": ubuntu_wsl2_docker_desktop_win_version_numbers
}
35 changes: 31 additions & 4 deletions docs/how-to/amdgpu-install.rst
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ Ubuntu

.. tab-set::
{% for (os_version, os_release) in config.html_context['ubuntu_version_numbers'] %}
.. tab-item:: Ubuntu {{ os_version }}
.. tab-item:: {{ os_version }}
:sync: ubuntu-{{ os_version}}

.. code-block:: bash
Expand All @@ -49,7 +49,7 @@ Red Hat Enterprise Linux

.. tab-set::
{% for (os_release, os_version) in config.html_context['rhel_version_numbers'] %}
.. tab-item:: RHEL {{ os_version }}
.. tab-item:: {{ os_version }}
:sync: rhel-{{ os_version }} rhel-{{ os_release }}

.. code-block:: bash
Expand All @@ -65,7 +65,7 @@ Oracle Linux

.. tab-set::
{% for (os_release, os_version) in config.html_context['ol_version_numbers'] %}
.. tab-item:: OL {{ os_version }}
.. tab-item:: {{ os_version }}
:sync: ol-{{ os_version }} ol-{{ os_release }}

.. code-block:: bash
Expand All @@ -82,14 +82,32 @@ SUSE Linux Enterprise

.. tab-set::
{% for os_version in config.html_context['sles_version_numbers'] %}
.. tab-item:: SLES {{ os_version }}
.. tab-item:: {{ os_version }}

.. code-block:: bash
:substitutions:

sudo zypper --no-gpg-checks install https://repo.radeon.com/amdgpu-install/|amdgpu_version|/sle/{{ os_version }}/amdgpu-install-|amdgpu_install_version|.noarch.rpm
{% endfor %}

Ubuntu (WSL2 / Docker for Windows)
--------------------------------------------------------------------

.. datatemplate:nodata::

.. tab-set::
{% for (os_version, os_release) in config.html_context['ubuntu_wsl2_docker_desktop_win_version_numbers'] %}
.. tab-item:: {{ os_version }}
:sync: ubuntu-wsl2-docker-desktop-win-{{ os_version}}

.. code-block:: bash
:substitutions:

sudo apt update
wget https://repo.radeon.com/amdgpu-install/|amdgpu_version|/ubuntu/{{ os_release }}/amdgpu-install_|amdgpu_install_version|_all.deb
sudo apt install ./amdgpu-install_|amdgpu_install_version|_all.deb
{% endfor %}

Use cases
=================================================

Expand Down Expand Up @@ -128,6 +146,8 @@ output below.
- HIP runtimes
- Machine learning framework
- All ROCm libraries and applications
wsl (for using ROCm in a WSL)
- ROCr WSL runtime library (Ubuntu 22.04 only)
rocmdev (for developers requiring ROCm runtime and
profiling/debugging tools)
- HIP runtimes
Expand Down Expand Up @@ -216,6 +236,13 @@ To install use cases specific to your requirements, use the installer

sudo amdgpu-install --usecase=rocm,asan

- To install the WSL2 and Docker Desktop for Windows compatible binaries add
``wsl``. For example:

.. code-block:: bash

sudo amdgpu-install --usecase=rocm,wsl --no-dkms --no-32

Uninstalling ROCm
=================================================

Expand Down
74 changes: 49 additions & 25 deletions docs/how-to/docker.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,54 +15,78 @@ ROCm kernel-mode driver must be installed on the host. Please refer to
user-space parts (like the HIP-runtime or math libraries) of the ROCm stack will
be loaded from the container image and don't need to be installed to the host.

Linux Docker containers running on Windows do not make use of ``amdgpu-dkms``,
but instead use an alternative user-space ROCm Runtime (ROCr) relaying to the
host driver.

.. _docker-access-gpus-in-container:

Accessing GPUs in containers
==========================================

In order to access GPUs in a container (to run applications using HIP, OpenCL or
OpenMP offloading) explicit access to the GPUs must be granted.
.. tab-set::

The ROCm runtimes make use of multiple device files:
.. tab-item:: Native Linux
:sync: native-linux-tab

- ``/dev/kfd``: the main compute interface shared by all GPUs
- ``/dev/dri/renderD<node>``: direct rendering interface (DRI) devices for each
GPU. Where ``<node>`` is a number for each card in the system starting from 128.
In order to access GPUs in a container (to run applications using HIP, OpenCL or
OpenMP offloading) explicit access to the GPUs must be granted.

Exposing these devices to a container is done by using the
`--device <https://docs.docker.com/engine/reference/commandline/run/#device>`_
option, i.e. to allow access to all GPUs expose ``/dev/kfd`` and all
``/dev/dri/renderD`` devices:
The ROCm runtimes make use of multiple device files:

.. code-block:: shell
- ``/dev/kfd``: the main compute interface shared by all GPUs
- ``/dev/dri/renderD<node>``: direct rendering interface (DRI) devices for each
GPU. Where ``<node>`` is a number for each card in the system starting from 128.

docker run --device /dev/kfd --device /dev/renderD128 --device /dev/renderD129 ...
Exposing these devices to a container is done by using the
`--device <https://docs.docker.com/engine/reference/commandline/run/#device>`_
option, i.e. to allow access to all GPUs expose ``/dev/kfd`` and all
``/dev/dri/renderD`` devices:

More conveniently, instead of listing all devices, the entire ``/dev/dri`` folder
can be exposed to the new container:
.. code-block:: shell

.. code-block:: shell
docker run --device /dev/kfd --device /dev/renderD128 --device /dev/renderD129 ...

More conveniently, instead of listing all devices, the entire ``/dev/dri`` folder
can be exposed to the new container:

.. code-block:: shell

docker run --device /dev/kfd --device /dev/dri

docker run --device /dev/kfd --device /dev/dri
Note that this gives more access than strictly required, as it also exposes the
other device files found in that folder to the container.

Note that this gives more access than strictly required, as it also exposes the
other device files found in that folder to the container.
.. tab-item:: WSL2 / Docker Desktop for Windows
:sync: wsl2-docker-desktop-win-tab

GPUs are automatically available to ROCr on WSL2 kernels and need no user intervention.

.. _docker-restrict-gpus:

Restricting a container to a subset of the GPUs
-------------------------------------------------------------------------------------------------

If a ``/dev/dri/renderD`` device is not exposed to a container then it cannot use
the GPU associated with it; this allows to restrict a container to any subset of
devices.
.. tab-set::

For example to allow the container to access the first and third GPU start it
like:
.. tab-item:: Native Linux
:sync: native-linux-tab

.. code-block:: shell
If a ``/dev/dri/renderD`` device is not exposed to a container then it cannot use
the GPU associated with it; this allows to restrict a container to any subset of
devices.

For example to allow the container to access the first and third GPU start it
like:

.. code-block:: shell

docker run --device /dev/kfd --device /dev/dri/renderD128 --device /dev/dri/renderD130 <image>

.. tab-item:: WSL2 / Docker Desktop for Windows
:sync: wsl2-docker-desktop-win-tab

docker run --device /dev/kfd --device /dev/dri/renderD128 --device /dev/dri/renderD130 <image>
Devices can't be filtered at the hypervisor level.

Additional options
-------------------------------------------------------------------------------------------------
Expand Down
2 changes: 2 additions & 0 deletions docs/how-to/native-install/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ Installation via native package manager
- :doc:`Ubuntu</how-to/native-install/ubuntu>`
- :doc:`Red Hat Enterprise Linux</how-to/native-install/rhel>`
- :doc:`SUSE Linux Enterprise</how-to/native-install/sles>`
- :doc:`Oracle Linux</how-to/native-install/ol>`
- :doc:`Ubuntu (WSL2 / Docker Desktop for Windows)</how-to/native-install/ubuntu-wsl2-docker-desktop-win>`

.. grid-item-card:: Upgrade

Expand Down
4 changes: 2 additions & 2 deletions docs/how-to/native-install/ol.rst
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ Register kernel-mode driver

.. tab-set::
{% for (os_release, os_version) in config.html_context['ol_version_numbers'] %}
.. tab-item:: OL {{ os_version }}
.. tab-item:: {{ os_version }}
:sync: ol-{{ os_version }} ol-{{ os_release }}

.. code-block:: bash
Expand All @@ -46,7 +46,7 @@ Register ROCm packages

.. tab-set::
{% for os_release in config.html_context['ol_release_version_numbers'] %}
.. tab-item:: OL {{ os_release }}
.. tab-item:: {{ os_release }}
:sync: ol-{{ os_release }}

.. code-block:: bash
Expand Down
6 changes: 6 additions & 0 deletions docs/how-to/native-install/post-install.rst
Original file line number Diff line number Diff line change
Expand Up @@ -62,3 +62,9 @@ Post-installation instructions
.. code-block:: bash

sudo zypper search --installed-only

.. tab-item:: Ubuntu (WSL2 / Docker for Windows)

.. code-block:: bash

sudo apt list --installed
4 changes: 2 additions & 2 deletions docs/how-to/native-install/rhel.rst
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ Register kernel-mode driver

.. tab-set::
{% for (os_release, os_version) in config.html_context['rhel_version_numbers'] %}
.. tab-item:: RHEL {{ os_version }}
.. tab-item:: {{ os_version }}
:sync: rhel-{{ os_version }} rhel-{{ os_release }}

.. code-block:: bash
Expand All @@ -46,7 +46,7 @@ Register ROCm packages

.. tab-set::
{% for os_release in config.html_context['rhel_release_version_numbers'] %}
.. tab-item:: RHEL {{ os_release }}
.. tab-item:: {{ os_release }}
:sync: rhel-{{ os_release }}

.. code-block:: bash
Expand Down
2 changes: 1 addition & 1 deletion docs/how-to/native-install/sles.rst
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ Register kernel-mode driver

.. tab-set::
{% for os_version in config.html_context['sles_version_numbers'] %}
.. tab-item:: SLES {{ os_version }}
.. tab-item:: {{ os_version }}

.. code-block:: bash
:substitutions:
Expand Down
Loading
Loading