CUSPARSE does not support reductions

[CarloLucibello]

On master, even the simplest reduction fails:

julia> using CUDA, CUDA.CUSPARSE, Random, SparseArrays

julia> x = sprand(3, 3, 0.5) |> CuSparseMatrixCSR
3×3 CuSparseMatrixCSR{Float64, Int32} with 4 stored entries:
  ⋅                  0.33280841492717694   ⋅ 
 0.2627012132720039  0.2749904007443871    ⋅ 
  ⋅                   ⋅                   0.009417688176282524

julia> sum(x)
ERROR: Scalar indexing is disallowed.
Invocation of getindex resulted in scalar indexing of a GPU array.
This is typically caused by calling an iterating implementation of a method.
Such implementations *do not* execute on the GPU, but very slowly on the CPU,
and therefore are only permitted from the REPL for prototyping purposes.
If you did intend to index this array, annotate the caller with @allowscalar.
Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:33
  [2] assertscalar(op::String)
    @ GPUArrays ~/.julia/packages/GPUArrays/umZob/src/host/indexing.jl:53
  [3] getindex(xs::CuArray{Int32, 1, CUDA.Mem.DeviceBuffer}, I::Int64)
    @ GPUArrays ~/.julia/packages/GPUArrays/umZob/src/host/indexing.jl:86
  [4] getindex(A::CuSparseMatrixCSR{Float64, Int32}, i0::Int64, i1::Int64)
    @ CUDA.CUSPARSE ~/.julia/packages/CUDA/BJT41/lib/cusparse/array.jl:312
  [5] _getindex
    @ ./abstractarray.jl:1262 [inlined]
  [6] getindex
    @ ./abstractarray.jl:1218 [inlined]
  [7] iterate
    @ ./abstractarray.jl:1144 [inlined]
  [8] iterate
    @ ./abstractarray.jl:1142 [inlined]
  [9] _foldl_impl
    @ ./reduce.jl:56 [inlined]
 [10] foldl_impl(op::Base.BottomRF{typeof(Base.add_sum)}, nt::Base._InitialValue, itr::CuSparseMatrixCSR{Float64, Int32})
    @ Base ./reduce.jl:48
 [11] mapfoldl_impl
    @ ./reduce.jl:44 [inlined]
 [12] #mapfoldl#244
    @ ./reduce.jl:162 [inlined]
 [13] mapfoldl
    @ ./reduce.jl:162 [inlined]
 [14] _mapreduce
    @ ./reduce.jl:423 [inlined]
 [15] _mapreduce_dim
    @ ./reducedim.jl:330 [inlined]
 [16] #mapreduce#725
    @ ./reducedim.jl:322 [inlined]
 [17] mapreduce
    @ ./reducedim.jl:322 [inlined]
 [18] #_sum#735
    @ ./reducedim.jl:894 [inlined]
 [19] _sum
    @ ./reducedim.jl:894 [inlined]
 [20] #_sum#734
    @ ./reducedim.jl:893 [inlined]
 [21] _sum
    @ ./reducedim.jl:893 [inlined]
 [22] #sum#732
    @ ./reducedim.jl:889 [inlined]
 [23] sum(a::CuSparseMatrixCSR{Float64, Int32})
    @ Base ./reducedim.jl:889
 [24] top-level scope
    @ REPL[49]:1
 [25] top-level scope
    @ ~/.julia/packages/CUDA/BJT41/src/initialization.jl:52

I get a similar errors with CuSparseMatrixCSC and sum(x; dims=1).

[maleadt]

Not going to happen anytime soon.

[CarloLucibello]

Workaround for sum:

Base.sum(x::AbstractCuSparseMatrix; dims=:) = cusparse_sum(x, Val(dims))

cusparse_sum(x, ::Val{:}) = sum(cusparse_sum(x, Val(1)))

function cusparse_sum(x::AbstractCuSparseMatrix, ::Val{1})
    m, n = size(x)
    v = CUDA.ones(1, m)
    return v * x
end

function cusparse_sum(x::AbstractCuSparseMatrix, ::Val{2})
    m, n = size(x)
    v = CUDA.ones(n, 1)
    return x * v
end

[maleadt]

Some support for reductions (mapreduce, i.e., not in-place yet; only dims=1, dims=2, dims=:) in #1406. Please take a look.