Unexpected type when constructing StateSpaceSet from existing StateSpaceSets

[kahaaga]

Describe the problem

With v2, we can no longer construct new StateSpaceSets with the correct type from existing datasets using horizontal concatenation. I discovered this because a bunch of tests upstream in Associations.jl broke when I now tried to up the compat version for StateSpaceSets.jl.

The only way to get the old behaviour is to first explicitly collect each column of each marginal dataset, and then create the joint space. This workaround works, but leads to tremendous allocations which, until a solution is in place, significantly slow down all algorithms upstream which depended on the old behaviour.

Potential solution

• Formally let horizontal concatenation of existing StateSpaceSets be a way of constructing them (update docstring to StateSpaceSet
• Require the user to specify the final container type if doing so (necessary, because individual sssets may have different inner container types).

I think this should be non-conflicting with the

Minimal Working Example

using StateSpaceSets # v2.1
julia> x, y = rand(100), rand(100); X, Y = StateSpaceSet(X), StateSpaceSet(Y); StateSpaceSet(X, Y)
2-dimensional StateSpaceSet{SVector{1, Float64}} with 50 points
 [0.282122]    [0.282122]
 [0.497362]    [0.497362]
 [-0.0778714]  [-0.0778714]
 [-0.209397]   [-0.209397]
 [-0.115629]   [-0.115629]
 [0.948698]    [0.948698]
 [-0.430675]   [-0.430675]
 [-0.104782]   [-0.104782]
 [1.07372]     [1.07372]
 [1.0829]      [1.0829]
 ⋮
 [0.146421]    [0.146421]
 [-0.207701]   [-0.207701]
 [-2.44547]    [-2.44547]
 [0.293089]    [0.293089]
 [0.668105]    [0.668105]
 [-0.188511]   [-0.188511]
 [-0.351139]   [-0.351139]
 [-1.03488]    [-1.03488]
 [-1.48079]    [-1.48079]

The resulting inner type is SVector{1, Float64}.

But, the result that I want is (which is also the pre v2 behaviour):

julia> StateSpaceSet(x, y)
2-dimensional StateSpaceSet{Float64} with 100 points
 0.355049   0.947321
 0.830712   0.395608
 0.118574   0.575198
 0.288244   0.869552
 0.553142   0.287584
 0.167979   0.50103
 0.991091   0.782378
 0.924755   0.283329
 0.339783   0.492456
 0.312601   0.418234
 ⋮
 0.430848   0.138478
 0.859612   0.787565
 0.122025   0.431661
 0.93248    0.0949152
 0.333865   0.772076
 0.0220937  0.500216
 0.739244   0.0262685
 0.525576   0.00749832
 0.80782    0.732381

[Datseris]

Okay that's easy, we just need one more constructor for Vararg{stateSpaceSet} that calls the hcat function internally.

It took me so much time to make hcat work hahaha so let's just call this one. Can you put together a pR?

[Datseris]

At the moment the docstring of StateSpaceSet does not even mention that you can construct it from given StateSpaceSets. It just takes then as Vector{X} and makes a 2-element vector of X. But X is already a vector :D

[kahaaga]

It just takes then as Vector{X} and makes a 2-element vector of X. But X is already a vector :D

Yes, that's the issue.

I frequently do things like

function some_func(x::AbstractStateSpaceSet, y::AbstractStateSpaceSet)
    z = StateSpaceSet(x, y)
end

where both x and y may be 2-dimensional or higher.

Can you put together a pR?

I'll do a PR. Which I kind of have to, because updating Associations.jl depends on this working properly (I don't want to introduce performance regressions, and using the workaround would require some time-consuming re-writing of some internals if not implemented)

At the moment the docstring of StateSpaceSet does not even mention that you can construct it from given StateSpaceSets.

I'll add that as part of the doctoring if I get it working.

[kahaaga]

Ah, this was as was easy as introducing StateSpaceSets(xs::AbstractStateSpaceSet) = hcat(xs...).

Then this works:

julia> X = StateSpaceSet(rand(10, 3)); Y = StateSpaceSet(rand(10, 4)); Z = StateSpaceSet(X, Y)
7-dimensional StateSpaceSet{Float64} with 10 points
 0.926997  0.53421   0.843038   0.735462  0.376864   0.844773   0.341259
 0.247685  0.329002  0.0977652  0.278606  0.189166   0.882888   0.0298962
 0.38986   0.4175    0.87285    0.325098  0.601125   0.724638   0.388005
 0.497634  0.656108  0.831676   0.988828  0.807545   0.850886   0.425285
 0.757449  0.948496  0.324714   0.960758  0.862568   0.0950807  0.53074
 0.504828  0.303373  0.312139   0.839023  0.11291    0.129833   0.467016
 0.241154  0.666729  0.272263   0.872927  0.689613   0.118522   0.9514
 0.09106   0.211788  0.492563   0.659997  0.0875356  0.329951   0.499794
 0.647615  0.309002  0.39649    0.657336  0.363963   0.0582504  0.942135
 0.5497    0.10165   0.47628    0.357906  0.160913   0.492181   0.921648

[kahaaga]

It'll be a bit more involved to make it work with specifying the containtertype (e.g. MVector). Can we just push this internal fix for now (which ensures that nothing breaks upstream), and then do a proper fix later?

I have very limited capacity to do anything significant work here now beyond fixing compatibility issues