Add build files

[K0IN]

Add build files (Docker), as stated in #31, i would like to use earthly to build binaries (docker does not seem to be the correct tool to do this).

We can also discuss switching, to make / cmake for a standardized build system (so people can hack more easily).

i also added a Dockerfile.client so people can use it as a base image for projects (switching out their base image with cuda client)

Lets also talk about:

• Automatic builds (on push)
• Providing prebuilds binaries / Docker images

[K0IN]

Serve must be launched with docker run --gpus all

[kevmo314]

Thanks for the config files! This looks like a good improvement towards productionization. One question on the published asset.

[kevmo314]

I think this client file is not quite as useful for anything outside testing as one would likely want to embed the .so file into one of their Dockerfiles instead of using this container as a base. Could we publish the .so file onto GitHub Releases for the client?

[K0IN]

my idea was to offer a drop-in replacement for local development, example you got a dev env and instead of using cuda as a base you can just drop in scuda.

I see your point, but i also see no harm in releasing this.

[K0IN]

But before we merge this please validate that it is working for you, i got some issues (but seems to be my host) since the (locally) build scuda does not work either.

[K0IN]

But before we merge this please validate that it is working for you, i got some issues (but seems to be my host) since the (locally) build scuda does not work either.

okay I got it working, but only on my local machine my plan was to rent a GPU machine on runpod and forward cuda to local, but I can't get it to work, I'm not sure if it is a connection issue tho(I'm still testing). on my Linux laptop it works (with docker/ my pre built binaries from GitHub).

But on my windows /wsl machine I can't get it to work,always returning invalid version even if I host scuda on the same machine.

[maltegrosse]

I tried to use the Dockerfiles to build it locally, but I needed to do some changes, e.g. install cudnn9-cuda-12, as I use cuda 12.2 on my host system, which forced me to downgrade to ubuntu 22.04

modified files can be found here:
https://github.com/maltegrosse/scuda/tree/dev

1. local test
   After successfully building, I ended up to first test it on the same GPU machine (Ubuntu x64 with working container toolkit)
   Running LD_PRELOAD=/cuda/libscuda.so nvidia-smi lead

scuda-server    | *** stack smashing detected ***: terminated
scuda-server exited with code 139

and client side shows an the normal nvidia-smi screen, but without any GPU

2. remote test
   I rebuilded the client container (mac os, arm64), running
   Running LD_PRELOAD=/cuda/libscuda.so nvidia-smi lead

Segmentation fault

on client side - but no activity on server side.

I tested the network functionality to do a curl to the $SCUDA_SERVER on port 14833, which shows some activity on the server - which means the address is available.

Any ideas?

[maltegrosse]

@kevmo314 Do you know if it makes any differences using hostnames instead of IP addresse on your side to target the server address?

[kevmo314]

@kevmo314 Do you know if it makes any differences using hostnames instead of IP addresse on your side to target the server address?

It should not, although we have been primarily testing with IP addresses. Are you able to get hostnames working outside the docker container?

[K0IN]

we need debug logging for this.

also: could you maybe try my earhtly build in my fork, and see if it works natively https://github.com/K0IN/scuda/blob/main/Earthfile

[maltegrosse]

I haven't used earthly yet. As we are on GitHub, we could just use github workflows?

[K0IN]

since earthly is repeatable and can run in GH actions with cross compilation and local so you got a build system that works on your machine and on GitHub, I would like to see if a locally built binary works for you

[maltegrosse]

I will give a try once I am back to work next year. Do you also have a working build in earthly for cuda 12.2 ?