Build w/o sparse

[bryantbiggs]

Given that the sparse implementation is now marked as deprecated, is there a way to build without sparse? make lib includes both, make sparse-lib only builds the sparse implementation; it would be great if there was a way to not build nor include any of the sparse implementation

[abdelfattah83]

Using make lib should indeed build the dense component only. I just tried a fresh build using Makefile.

[bryantbiggs]

hmm, perhaps I have mis-configured something because when I use make lib, its generating both libmagma.so and libmagma_sparse.so

From the container generated using the Dockerfile provided below:

root@aff53b3853ec:/usr/local/magma/lib# ls -la
total 972084
drwxr-xr-x 3 root root      4096 Jan  6 21:54 .
drwxr-xr-x 4 root root      4096 Jan  6 21:54 ..
-rw-r--r-- 1 root root 936313712 Jan  6 21:24 libmagma.so
-rw-r--r-- 1 root root  59080616 Jan  6 21:41 libmagma_sparse.so
drwxr-xr-x 2 root root      4096 Jan  6 21:54 pkgconfig

Dockerfile used to build the lib(s)

# syntax=docker/dockerfile:1
FROM ubuntu:24.04 AS base

# nvidia-container-runtime
ENV NVIDIA_DRIVER_CAPABILITIES=compute,utility

ARG CUDA_MAJOR_VERSION='12'
ARG CUDA_MINOR_VERSION='6'

RUN <<EOT
  rm -f /etc/apt/apt.conf.d/docker-clean
  echo 'Binary::apt::APT::Keep-Downloaded-Packages "true";' > /etc/apt/apt.conf.d/keep-cache
  echo 'APT::Install-Suggests "0";' >> /etc/apt/apt.conf.d/00-docker
  echo 'APT::Install-Recommends "0";' >> /etc/apt/apt.conf.d/00-docker
EOT

RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
    --mount=type=cache,target=/var/lib/apt,sharing=locked <<EOT

  apt update
  apt upgrade -y
  apt install -y \
    ca-certificates \
    curl \
    gnupg2

  curl -fsSLO https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64/cuda-keyring_1.1-1_all.deb
  dpkg -i cuda-keyring_1.1-1_all.deb
  rm cuda-keyring_1.1-1_all.deb
EOT

RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
    --mount=type=cache,target=/var/lib/apt,sharing=locked <<EOT

  apt update
  apt install -y \
    cmake \
    cuda-toolkit-${CUDA_MAJOR_VERSION}-${CUDA_MINOR_VERSION} \
    git \
    libopenblas-openmp-dev \
    libopenblas64-0-openmp \
    python3.12 \
    python3-pip \
    python-is-python3
EOT

ENV CXX=g++
ENV CC=gcc

WORKDIR /tmp

RUN <<EOT
  git clone --branch=master --depth=1 https://github.com/icl-utk-edu/magma.git
EOT

RUN <<EOT
  cat <<-EOF > /tmp/magma/make.inc
BACKEND    = cuda
GPU_TARGET = Hopper
FORT       = false
EOF

  cd /tmp/magma
  make generate

  mkdir build
  cd build
  cmake \
      -DUSE_FORTRAN=OFF \
      -DCMAKE_CUDA_COMPILER=/usr/local/cuda/bin/nvcc \
      -DGPU_TARGET='Hopper' \
      -DMAGMA_ENABLE_CUDA=ON \
      ..

  make lib -j $(nproc)
  make install
EOT

[nbeams]

Hi @bryantbiggs, I think you might need to skip the explicit make generate step. make generate generates code for both dense and sparse. However, if you just do make lib in a clean directory (fresh checkout or after make cleangen), it should auto-generate and build only the dense library.

[bryantbiggs]

Hmm, I'm a bit confused. I thought we have to run make generate if we are checking out directly from source instead of pulling down a generated release per the docs on the README

Step 0: setup
    If you downloaded an official release (e.g., magma-2.6.0.tar.gz), you can
    skip this step.
    If you checked out MAGMA from bitbucket, you first need to generate all
    the precisions. Currently this is done only by the Makefile, not by CMake.
    Using a minimal make.inc configuration (with BACKEND=hip or cuda), e.g.,
        echo -e 'BACKEND = hip\nFORT = true' > make.inc
        make generate
    That should run `python tools/codegen.py` on all the src files and
    create CMake.src.{hip|cuda}

We are pulling directly from source due to the various issues related to CUDA 12.6 - #7, #12

I get errors if I remove the make generate because the generated files do not exist yet and therefore unable to build

[nbeams]

Oh, yes, you are using the CMake build. We typically recommend using regular make if possible, and we generally use it ourselves -- so I forgot that the CMake build targets don't work the same way. Sorry! make lib works to generate/build just the dense library if you are using the Makefile build only. We should probably adjust the CMake build so that the build commands are consistent with the Makefile behavior -- thank you for bringing this to our attention!

For now, if you want to skip building sparse, and you definitely need to stick with CMake, you could hack the CMakeLists.txt:

• comment out/delete lines 582 - 666 (starts with # compile MAGMA sparse library)
• remove magma_sparse from the install list in line 671

Then, I think make lib should build just the dense library. I tried it just now and it seems to work, though I didn't do the install step...