Deep Multi-Fidelity Gaussian Processes in gpytorch

[filippobrand]

Hi everybody!
I'm trying to implement the Deep Multi-Fidelity Gaussian Process procedure found in this paper.
To do so I've pretty much mixed two tutorials (DKL and Hadamard) and what I found in this issue here on GitHub, which apparently worked on previous versions of gpytorch but doesn't now. Right now I have two scripts:

• The first one kind of works, but I can't get reliably good results (compared to those on the aforementioned paper) and I don't fully understand if there is something wrong with my code or if I'm missing something. Playing around with the learning ratio and optimizer doesn't seem to improve the accuracy. In this case I'm training the NN/GP on the full database, without using any dataloader
• The second one is a more direct copy of the cited one and makes use of a dataloader, but I keep running into various error. Right now I'm stuck at "You must train on the same training inputs!" or something like that, and I don't understand what am I doing wrong. This script uses a dataloader and trains input in batches.

I'd be very glad if anyone has come across the same problems and has any advice/guidance. Also please bear in mind that I'm just starting to learn python and the theory behind GPs, so I might have missed some basic stuff.

Thanks in advance!

[AkrhamHorrorComing]

Hi. I'm copying the paper (Nonlinear information fusion algorithms for data-efficient multi-fidelity modelling), but also can't get the exact result as the example code coded by GPy, which is published by the paper author. I think the problem may be occured in the model trainning process. In the origin code, the author restart the tranning process by using GPy.optimize_restarts. If u have some other advises, welcome to reply me. Thx~

[AkrhamHorrorComing]

Maybe i've found the reason. This answer can be the reference for the problem:https://github.com/cornellius-gp/gpytorch/issues.
As the result, we need to restrart the optimizer handly to ensure the accuracy of hyperparameters.