Using non-tabular datasets, like sequences and grids, in PySR

[MilesCranmer]

Moving from #323:

@puja93: Hi @MilesCranmer... Thanks for the great paper and repo.

Regarding the 8th point in your paper :

Do you have any links for documentation or tutorial on that ? Especially for the sequential and grids dataset.

Thanks in advance

[MilesCranmer]

Hi @puja93,

Good question! So the primary recommended way to solve this is via PySR's integration with deep learning frameworks. For example, see the last part of the example colab notebook here: https://github.com/MilesCranmer/symbolic_deep_learning for a time series example.

The other way to solve this to create a custom loss function that incorporates your dataset structure. For example, you can learn recursive functions like #540 (comment) which could be used for time series data, or unpack the data explicitly as in #426 (comment).

Vectors and matrices will soon be supported in the symbolic expressions themselves (see SymbolicML/DynamicExpressions.jl#85 for progress) – though the API for this isn't yet worked out. So for right now the above two options.

Cheers,
Miles

[puja93]

Please correct me if i'm wrong @MilesCranmer

What i understand is, we can eventually get either :

F(x_1, x_2, ..., x_m) = (y_1, y_2, ..., y_n) OR F(x_1, x_2, ..., x_m) = z

With m = number of inputs, n = number of logprob results, z is the final output

Which then implies, we could find analytical formula for

Fixed Input size - Fixed output size NN model (CNN, Vanilla DNN) / Grid & Tabular
Arbitrary input size - Fixed output size NN model (Transformer) / Sequence

[MilesCranmer]

Sorry I'm not sure I understand, could you try explaining what you mean by the difference between those two options?

[puja93]

Hi, sorry to reply late. Let me tell from another perspective :

I have a vanilla NN model based on tabular data, that output prediction real value (0 to 1). For the sake of the arguman let's say its 6 hidden layers deep, and 10 neurons in each layer. With one output layer. and 10 inputs.

What i want is to get an analytical formula replacing the model, which i'm still not sure how to do even with the notebook you've given. Is there any notebook example for at least 3 layers deep NN model ? Thanks @MilesCranmer

Oh and thanks for LLM - SR paper as well, would like to see more of this...