Type constructor in broadcast doesn't compile

[bicycle1885]

Describe the bug

The @. broadcast syntax doesn't work if it involves like Float32(2) in the expression.

To reproduce

The Minimal Working Example (MWE) for this bug:

julia> using CUDA

julia> x = randn(Float32, 3)
3-element Vector{Float32}:
  0.47644928
  3.393554
 -1.519229

julia> @. Float32(2) * x  # if x is on GPU, it works :)
3-element Vector{Float32}:
  0.95289856
  6.787108
 -3.038458

julia> x = cu(x)
3-element CuArray{Float32, 1, CUDA.Mem.DeviceBuffer}:
  0.47644928
  3.393554
 -1.519229

julia> @. Float32(2) * x  # but if x is on GPU, it won't work :(
ERROR: GPU compilation of kernel #broadcast_kernel#28(CUDA.CuKernelContext, CuDeviceVector{Float32, 1}, Base.Broadcast.Broadcasted{CUDA.CuArrayStyle{1}, Tuple{Base.OneTo{Int64}}, typeof(*), Tuple{Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{0}, Nothing, Type{Float32}, Tuple{Int64}}, Base.Broadcast.Extruded{CuDeviceVector{Float32, 1}, Tuple{Bool}, Tuple{Int64}}}}, Int64) failed
KernelError: passing and using non-bitstype argument

Argument 4 to your kernel function is of type Base.Broadcast.Broadcasted{CUDA.CuArrayStyle{1}, Tuple{Base.OneTo{Int64}}, typeof(*), Tuple{Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{0}, Nothing, Type{Float32}, Tuple{Int64}}, Base.Broadcast.Extruded{CuDeviceVector{Float32, 1}, Tuple{Bool}, Tuple{Int64}}}}, which is not isbits:
  .args is of type Tuple{Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{0}, Nothing, Type{Float32}, Tuple{Int64}}, Base.Broadcast.Extruded{CuDeviceVector{Float32, 1}, Tuple{Bool}, Tuple{Int64}}} which is not isbits.
    .1 is of type Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{0}, Nothing, Type{Float32}, Tuple{Int64}} which is not isbits.
      .f is of type Type{Float32} which is not isbits.

Manifest.toml

Paste your Manifest.toml here, or accurately describe which version of CUDA.jl and its dependencies (GPUArrays.jl, GPUCompiler.jl, LLVM.jl) you are using.

  [052768ef] CUDA v4.0.1
  [1af6417a] CUDA_Runtime_Discovery v0.1.1
  [0c68f7d7] GPUArrays v8.6.2
  [46192b85] GPUArraysCore v0.1.3
  [61eb1bfa] GPUCompiler v0.17.2
  [929cbde3] LLVM v4.16.0

Expected behavior

This might be an intended behavior but this is confusing because the syntax works perfectly on CPU.
The actual code in my program is much more complicated and it wasn't the easiest task to nail down.

Version info

julia> versioninfo()
Julia Version 1.8.5
Commit 17cfb8e65ea (2023-01-08 06:45 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 16 × Intel(R) Xeon(R) Gold 6134 CPU @ 3.20GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-13.0.1 (ORCJIT, skylake-avx512)
  Threads: 1 on 16 virtual cores
Environment:
  LD_LIBRARY_PATH = /home/kenta/.local/lib
  JULIA_PROJECT = @.

julia> CUDA.versioninfo()
CUDA runtime 11.8, artifact installation
CUDA driver 11.8
NVIDIA driver 515.43.4, originally for CUDA 11.7

Libraries:
- CUBLAS: 11.11.3
- CURAND: 10.3.0
- CUFFT: 10.9.0
- CUSOLVER: 11.4.1
- CUSPARSE: 11.7.5
- CUPTI: 18.0.0
- NVML: 11.0.0+515.43.4

Toolchain:
- Julia: 1.8.5
- LLVM: 13.0.1
- PTX ISA support: 3.2, 4.0, 4.1, 4.2, 4.3, 5.0, 6.0, 6.1, 6.3, 6.4, 6.5, 7.0, 7.1, 7.2
- Device capability support: sm_35, sm_37, sm_50, sm_52, sm_53, sm_60, sm_61, sm_62, sm_70, sm_72, sm_75, sm_80, sm_86

1 device:
  0: Quadro RTX 6000 (sm_75, 22.667 GiB / 24.000 GiB available)

[maleadt]

Simpler MWE:

julia> CUDA.rand(1) .+ Float32(2)
1-element CuArray{Float32, 1, CUDA.Mem.DeviceBuffer}: 2.1238053

julia> CUDA.rand(1) .+ Float32.(2)
ERROR: GPU compilation of kernel #broadcast_kernel#28(CUDA.CuKernelContext, CuDeviceVector{Float32, 1}, Base.Broadcast.Broadcasted{CUDA.CuArrayStyle{1}, Tuple{Base.OneTo{Int64}}, typeof(+), Tuple{Base.Broadcast.Extruded{CuDeviceVector{Float32, 1}, Tuple{Bool}, Tuple{Int64}}, Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{0}, Nothing, Type{Float32}, Tuple{Int64}}}}, Int64) failed
KernelError: passing and using non-bitstype argument

Argument 4 to your kernel function is of type Base.Broadcast.Broadcasted{CUDA.CuArrayStyle{1}, Tuple{Base.OneTo{Int64}}, typeof(+), Tuple{Base.Broadcast.Extruded{CuDeviceVector{Float32, 1}, Tuple{Bool}, Tuple{Int64}}, Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{0}, Nothing, Type{Float32}, Tuple{Int64}}}}, which is not isbits:
  .args is of type Tuple{Base.Broadcast.Extruded{CuDeviceVector{Float32, 1}, Tuple{Bool}, Tuple{Int64}}, Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{0}, Nothing, Type{Float32}, Tuple{Int64}}} which is not isbits.
    .2 is of type Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{0}, Nothing, Type{Float32}, Tuple{Int64}} which is not isbits.
      .f is of type Type{Float32} which is not isbits.

I'm not sure we can fix this in generality. We already have a workaround for broadcasting just the type constructor, but I'm not sure how to rewrite that when it's part of the broadcast:

CUDA.jl/src/broadcast.jl

Lines 17 to 19 in e9833ed

	# broadcasting type ctors isn't GPU compatible
	Broadcast.broadcasted(::CuArrayStyle{N}, f::Type{T}, args...) where {N, T} =
	Broadcasted{CuArrayStyle{N}}((x...) -> T(x...), args, nothing)

[N5N3]

Perhaps we can fix this within adapt (by transforming bc.f from a Type into a function):

function Adapt.adapt_structure(::Adaptor, bc::Broadcast.Broadcasted{Style}) where {Style}
    Broadcasted{Style}(adapt(Adaptor(), Base.maybeconstructor(bc.f)), adapt(Adaptor(), bc.args), bc.axes)
end

[BioTurboNick]

I know this is closed, but is it maybe possible to extend the error message? Can hitting this issue be detected and a more specific and helpful message be produced?

[simonbyrne]

This specific problem shouldn't occur any more.

[BioTurboNick]

Oh. Well, it seems to have, or something quite similar. Trying to construct a matrix of normal distributions with Distributions.Normal.(means, stds) produces this error during pullback from within Zygote. Spent a good day trying to figure out why it was having trouble with the broadcast.

[BioTurboNick]

Hmm, may be related, made a new issue: #2514