[Question] How Likelihood is related to model.likelihood?

[mohammad-saber]

Hi, thank you for reading my question.

I have a question about training process explained in the documentation.

After model fitting, I can get model prediction by:

with torch.no_grad(), gpytorch.settings.fast_pred_var():
    test_x = torch.linspace(0, 1, 51)
    observed_pred = likelihood(model(test_x))

I was wondering how likelihood is updated during model training. When I compare model.likelihood.noise.item()& likelihood.noise.item(), they have equal values.

It seems that likelihood is pointed to model.likelihood.

[patel-zeel]

Hi @mohammad-saber,

Yes, you are correct that both of them are pointing to the same object. I suppose you are wondering how it is updated during training despite we do not do output = likelihood(model(train_x)) but instead we do output = model(train_x). Note that, we have already passed likelihood to the model definition in the following code block:

# We will use the simplest form of GP model, exact inference
class ExactGPModel(gpytorch.models.ExactGP):
    def __init__(self, train_x, train_y, likelihood):
        super(ExactGPModel, self).__init__(train_x, train_y, likelihood)
        self.mean_module = gpytorch.means.ConstantMean()
        self.covar_module = gpytorch.kernels.ScaleKernel(gpytorch.kernels.RBFKernel())

    def forward(self, x):
        mean_x = self.mean_module(x)
        covar_x = self.covar_module(x)
        return gpytorch.distributions.MultivariateNormal(mean_x, covar_x)

# initialize likelihood and model
likelihood = gpytorch.likelihoods.GaussianLikelihood()
model = ExactGPModel(train_x, train_y, likelihood)

Thus, likelihood is handled internally during the training (to be exact, here). Though, I am not sure why the authors chose this way because in my opinion, output = likelihood(model(train_x)) would have been cleaner. Maybe @gpleiss can guide us on practical reasons for not making it explicit.

Edit: I think likelihood(model(x)) is not used during training to make training more generic. In case one is using an external routine for training, the current implementation will work without any problems (we can simply pass the model and input, outputs to the routine like any other torch model).