Possibilities wrt. intel GPUs (et. al.); SYCL / OpenCL / Vulkan etc.?

[ghchris2021]

Possibilities wrt. intel GPUs (et. al.); SYCL / OpenCL / Vulkan etc.?

I saw one commit about adding some SYCL support though I'm not sure whether it was partially or fully working at that time or presently.

Personally my LLM inference use case interests are in the inference enablement of distributed (few LAN connected desktops) inference of popular LLMs across systems having some mix of CPU+RAM / NVIDIA GPUs / Intel consumer DGPUs (ARC7 et. seq.).

So far I haven't found any inference platform that handles such a case or definitively plans to any time soon; I think maybe llama.cpp is getting close, from what I see about aphrodite and intel / amd GPU support it seems hopeful perhaps there are some upcoming possibilities. So I thought I'd inquire.

Thank you very much!

[AlpinDale]

Intel support has been added in the rc_054 branch via Intel Extensions for PyTorch. Please check it out there. I will be adding proper documentation with the next release, but you can use the dockerfile provided.