[Question] Multivariate Gaussian Process Time-Series Forecasting

[eleGANghini]

Hello everyone,

I am trying to find a GPyTorch implementation example for multivariate time-series forecasting problem.
As implemented in the link below:
https://scikit-learn.org/stable/auto_examples/gaussian_process/plot_gpr_co2.html
forecasting approach can be seen in scikit-learn.
In order to have a similar result what should one do? Currently, I am employing ExactGP Regression.

Let's imagine we have (N+M, 5) data with N+M entry and 5 features with corresponding (N+M, 1) labels.
For instance, I am planning to feed (N, 5)[observed time-series] with its labels (N, 1); then I need to forecast (M, 1)[unobserved time-series] labels. I could not come up with a strategy to forecast future labels within the ExactGP/GPyTorch setting.

Any help/support will be much appreciated!

[wjmaddox]

You should just be able to adapt the gpytorch tutorial: https://docs.gpytorch.ai/en/stable/examples/01_Exact_GPs/Simple_GP_Regression.html for multiple input dimensions.

Then at test time, I'd imagine that you would simply update the time inputs for forecasting.

[eleGANghini]

I followed one of the examples, sure; namely, spectral mixture kernel notebook where I receive:
RuntimeError: output with shape [10, 1, 1] doesn't match the broadcast shape [10, 1, 8] when I try to feed multiple input dims and call initialize_from_data(train_x, train_y). Num_mixtures = 10, DS do have 8 features.

[eleGANghini]

I forgot to specify smk = SpectralMixtureKernel(num_mixtures, ard_num_dims=8)

[eleGANghini]

After running several experiments regarding multiple input dimensions, I would like to consult regarding a technical issue.
As I mentioned above, I use a dataset with 5 features and a corresponding label (each feature and the label is in a time-series format).
Let's say length of the time-series is L.
During forecasting, I provide a test set with observation length 0.6*L (observed part of the data, rest will be forecasting).

In order to receive corresponding label (or trajectory) for the remaining 0.4*L, it is expected for me to provide unobserved/unseen features from the future in an offline manner. As far as I know, multidimensional GP expects data with N=5 features unlike the univariate case.
I am wondering, what would be the proper real-time forecasting strategy, since I am unable feed unobserved features?

[wjmaddox]

If you don't have the observed features at test time, you probably want to figure out a way to get those features whether via observation or modelling.

For modelling, I mean something like the following approach:

• Model each feature using time (assuming that you have 5 features + a time variable) with a new model (call it model A).
• use all 5 features on the main model, model B.
• at time t+1, use model A to generate the input features for model B (whether via posterior sampling or the predictive mean).

Model A doesn't necessarily have to be a GP but could be :) .