useful docker image that can run on windows

[majerus1223]

Looking for a docker image that can run on windows just flat out. Struggling with the docs on windows, and this would make it much easier..

docker run -it --net=bridge --device=/dev/dri -p 11434:11434 -v E:\AI_Models:/root/.ollama/models -e PATH=/llm/ollama:$PATH -e OLLAMA_HOST=0.0.0.0 -e no_proxy=localhost,127.0.0.1 -e ZES_ENABLE_SYSMAN=1 -e OLLAMA_INTEL_GPU=true -e ONEAPI_DEVICE_SELECTOR=level_zero:0 -e DEVICE=Arc --shm-size="16g" --memory="32G" --name=ipex-llm intelanalytics/ipex-llm-xpu:latest

Always fails.

[glorysdj]

Could you please share the failure's output?
Please refer to https://github.com/intel/ipex-llm/blob/main/docs/mddocs/DockerGuides/docker_windows_gpu.md
and https://github.com/intel/ipex-llm/blob/main/docs/mddocs/DockerGuides/docker_pytorch_inference_gpu.md

[majerus1223]

PS C:\Users\james> docker run -it --net=bridge --device=/dev/dri -p 11434:11434 -v E:\AI_Models:/root/.ollama/models -e PATH=/llm/ollama:$PATH -e OLLAMA_HOST=0.0.0.0 -e no_proxy=localhost,127.0.0.1 -e ZES_ENABLE_SYSMAN=1 -e OLLAMA_INTEL_GPU=true -e ONEAPI_DEVICE_SELECTOR=level_zero:0 -e DEVICE=Arc --shm-size="16g" --memory="32G" --name=ipex-llm intelanalytics/ipex-llm-xpu:latest

bash: lesspipe: command not found
bash: dircolors: command not found
root@8ee77695ec28:/llm#```

[majerus1223]

Could you please share the failure's output? Please refer to https://github.com/intel/ipex-llm/blob/main/docs/mddocs/DockerGuides/docker_windows_gpu.md and https://github.com/intel/ipex-llm/blob/main/docs/mddocs/DockerGuides/docker_pytorch_inference_gpu.md

Posted the errors above. The docs are confusing with WSL. If I am running something like docker desktop, or rancher desktop why do I need to mess with wsl? Shouldnt I just be able to run the container with a few args from powershell and it just work? Trying to setup conda, and other env things ads to the confusion.

[majerus1223]

Finally making progress..

root@83359eb27245:/llm# sycl-ls
INFO: Output filtered by ONEAPI_DEVICE_SELECTOR environment variable, which is set to level_zero:0.
To see device ids, use the --ignore-device-selectors CLI option.

[level_zero:gpu] Intel(R) Level-Zero, Intel(R) Graphics [0x56a0] 1.6 [1.3.31294]
root@83359eb27245:/llm# sycl-ls --ignore-device-selectors
[opencl:cpu][opencl:0] Intel(R) OpenCL, AMD Ryzen 9 7900X 12-Core Processor             OpenCL 3.0 (Build 0) [2024.18.7.0.11_160000]
[opencl:gpu][opencl:1] Intel(R) OpenCL Graphics, Intel(R) Graphics [0x56a0] OpenCL 3.0 NEO  [24.39.31294.12]
[level_zero:gpu][level_zero:0] Intel(R) Level-Zero, Intel(R) Graphics [0x56a0] 1.6 [1.3.31294]

[majerus1223]

Still no luck, not really sure what the deal is, see a lot of errors like .

time=2025-01-27T02:03:42.448+08:00 level=INFO source=gpu.go:221 msg="looking for compatible GPUs"
time=2025-01-27T02:03:42.449+08:00 level=WARN source=gpu.go:732 msg="unable to locate gpu dependency libraries"
time=2025-01-27T02:03:42.449+08:00 level=WARN source=gpu.go:732 msg="unable to locate gpu dependency libraries"
time=2025-01-27T02:03:42.449+08:00 level=WARN source=gpu.go:732 msg="unable to locate gpu dependency libraries"
[GIN] 2025/01/27 - 02:03:42 | 200 |      33.894µs |       127.0.0.1 | HEAD     "/"
[GIN] 2025/01/27 - 02:03:42 | 200 |  107.286621ms |       127.0.0.1 | POST     "/api/show"
time=2025-01-27T02:03:44.343+08:00 level=WARN source=gpu.go:732 msg="unable to locate gpu dependency libraries"
time=2025-01-27T02:03:46.334+08:00 level=WARN source=gpu.go:732 msg="unable to locate gpu dependency libraries"
SIGSEGV: segmentation violation
PC=0x7fb5db35d0bc m=14 sigcode=1 addr=0x200000000
signal arrived during cgo execution

goroutine 25 gp=0xc000582c40 m=14 mp=0xc00058c808 [syscall]:
runtime.cgocall(0xfa9920, 0xc00005a748)
        /root/go/pkg/mod/golang.org/[email protected]/src/runtime/cgocall.go:157 +0x4b fp=0xc00005a720 sp=0xc00005a6e8 pc=0x40ceab
ollama/discover._Cfunc_oneapi_init(0x7fb548000b70, 0xc0003bac40)

[majerus1223]

@glorysdj This is the output from inside of WSL. Couple questions, I am running an AMD cpu would that matter? Is there any specific WSL configuration I need to permit the gpu to be accessed within wsl?

majerus@7900x:/mnt/c/Users/james$ sudo lshw -C display
[sudo] password for majerus:
  *-display:0
       description: 3D controller
       product: Microsoft Corporation
       vendor: Microsoft Corporation
       physical id: 2
       bus info: pci@191e:00:00.0
       version: 00
       width: 32 bits
       clock: 33MHz
       capabilities: bus_master cap_list
       configuration: driver=dxgkrnl latency=0
       resources: irq:0
  *-display:1
       description: 3D controller
       product: Microsoft Corporation
       vendor: Microsoft Corporation
       physical id: 4
       bus info: pci@6ab7:00:00.0
       version: 00
       width: 32 bits
       clock: 33MHz
       capabilities: bus_master cap_list
       configuration: driver=dxgkrnl latency=0
       resources: irq:0
majerus@7900x:/mnt/c/Users/james$ lspci
191e:00:00.0 3D controller: Microsoft Corporation Device 008e
4dd4:00:00.0 System peripheral: Red Hat, Inc. Virtio file system (rev 01)
6ab7:00:00.0 3D controller: Microsoft Corporation Device 008e
de3f:00:00.0 SCSI storage controller: Red Hat, Inc. Virtio console (rev 01)

[majerus1223]

majerus@7900x:/mnt/c/Users/james$ glxinfo -B
name of display: :0
display: :0  screen: 0
direct rendering: Yes
Extended renderer info (GLX_MESA_query_renderer):
    Vendor: Microsoft Corporation (0xffffffff)
    Device: D3D12 (AMD Radeon(TM) Graphics) (0xffffffff)
    Version: 23.2.1
    Accelerated: yes
    Video memory: 32816MB
    Unified memory: yes
    Preferred profile: core (0x1)
    Max core profile version: 4.2
    Max compat profile version: 4.2
    Max GLES1 profile version: 1.1
    Max GLES[23] profile version: 3.1
OpenGL vendor string: Microsoft Corporation
OpenGL renderer string: D3D12 (AMD Radeon(TM) Graphics)
OpenGL core profile version string: 4.2 (Core Profile) Mesa 23.2.1-1ubuntu3.1~22.04.3
OpenGL core profile shading language version string: 4.20
OpenGL core profile context flags: (none)
OpenGL core profile profile mask: core profile

OpenGL version string: 4.2 (Compatibility Profile) Mesa 23.2.1-1ubuntu3.1~22.04.3
OpenGL shading language version string: 4.20
OpenGL context flags: (none)
OpenGL profile mask: compatibility profile

OpenGL ES profile version string: OpenGL ES 3.1 Mesa 23.2.1-1ubuntu3.1~22.04.3
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.10

[majerus1223]

May have just gotten it to work, by disabling the Radeon adapter in windows. Testing now..

majerus@7900x:/mnt/c/Users/james$ glxinfo | grep "OpenGL renderer string"
OpenGL renderer string: D3D12 (AMD Radeon(TM) Graphics)
majerus@7900x:/mnt/c/Users/james$ exit
logout
PS C:\Users\james> wsl --shutdown
PS C:\Users\james> wsl
majerus@7900x:/mnt/c/Users/james$ glxinfo | grep "OpenGL renderer string"
OpenGL renderer string: D3D12 (Intel(R) Arc(TM) A770 Graphics)

[majerus1223]

Same issue after disabling and restarting..

PS C:\Users\james> docker exec -it ipex-llm bash
root@b184b9488e7b:/llm# sycl-ls
INFO: Output filtered by ONEAPI_DEVICE_SELECTOR environment variable, which is set to level_zero:0.
To see device ids, use the --ignore-device-selectors CLI option.                                                                                                                                                                                [level_zero:gpu] Intel(R) Level-Zero, Intel(R) Graphics [0x56a0] 1.6 [1.3.31294]

[GIN-debug] HEAD   /api/version              --> ollama/server.(*Server).GenerateRoutes.func2 (5 handlers)
time=2025-01-27T04:02:31.369+08:00 level=INFO source=gpu.go:221 msg="looking for compatible GPUs"
time=2025-01-27T04:02:31.369+08:00 level=WARN source=gpu.go:732 msg="unable to locate gpu dependency libraries"
time=2025-01-27T04:02:31.371+08:00 level=WARN source=gpu.go:732 msg="unable to locate gpu dependency libraries"
time=2025-01-27T04:02:31.371+08:00 level=WARN source=gpu.go:732 msg="unable to locate gpu dependency libraries"
[GIN] 2025/01/27 - 04:02:31 | 200 |     624.331µs |       127.0.0.1 | HEAD     "/"
[GIN] 2025/01/27 - 04:02:31 | 200 |  142.327141ms |       127.0.0.1 | POST     "/api/show"
time=2025-01-27T04:02:33.414+08:00 level=WARN source=gpu.go:732 msg="unable to locate gpu dependency libraries"
time=2025-01-27T04:02:35.498+08:00 level=WARN source=gpu.go:732 msg="unable to locate gpu dependency libraries"
SIGSEGV: segmentation violation
PC=0x7f0d18d5d0bc m=13 sigcode=1 addr=0x100000000
signal arrived during cgo execution

goroutine 27 gp=0xc000185c00 m=13 mp=0xc000346008 [syscall]:
runtime.cgocall(0xfa9920, 0xc00005a748)
        /root/go/pkg/mod/golang.org/[email protected]/src/runtime/cgocall.go:157 +0x4b fp=0xc00005a720 sp=0xc00005a6e8 pc=0x40ceab
ollama/discover._Cfunc_oneapi_init(0x7f0c80000b70, 0xc0002e0150)
        _cgo_gotypes.go:646 +0x3f fp=0xc00005a748 sp=0xc00005a720 pc=0x8131ff
ollama/discover.loadOneapiMgmt.func2(0x7f0c80000b70, 0xc0002e0150)
        /home/runner/_work/llm.cpp/llm.cpp/ollama-internal/discover/gpu.go:664 +0x4a fp=0xc00005a778 sp=0xc00005a748 pc=0x81a6aa

[majerus1223]

Its finally running now...

docker run -itd --net=bridge --device=/dev/dri --privileged -v E:\AI_Models:/root/.ollama/models -v /usr/lib/wsl:/usr/lib/wsl -e no_proxy=localhost,127.0.0.1 --memory="32G" -e DEVICE=Arc --shm-size="16g" --name=ipex-llm intelanalytics/ipex-llm-inference-cpp-xpu:latest

I then had to exec into the container and run the following

PS C:\Users\james> docker exec -it ipex-llm bash
root@b06a1867bb09:/llm# cd /llm/scripts/
root@b06a1867bb09:/llm/scripts# source ipex-llm-init --gpu --device $DEVICE

root@b06a1867bb09:/llm/scripts# bash start-ollama.sh

So why is all this required is there a more straight forward way to make this just work ?