Thin-plate kernel, NaNs as variance

[aspirinium]

Hi,

I am implementing a thin-plate kernel for a 2D space, as described in [http://gpss.cc/gpip/abstract/owilliams.pdf](this paper). However, although the estimates for the mean function seem to be correct, the output of observed_pred.variance is tensor([nan, nan, nan, ..., nan, nan, nan]). I've been playing around with different parameters, but couldn't resolve the problem and hoped I could gain some insights here.

My implementation of the kernel is pretty straightforward:

class ThinPlateKernel(gpytorch.kernels.Kernel):

    def forward(self, x1, x2, **params):
        r = self.covar_dist(x1, x2, **params)
        r.where(r == 0, torch.as_tensor(1e-20))
        R = r.max()
        cov = (2 * r ** 2 * torch.log(r.abs()) - (1 + 2 * torch.log(R)) * r ** 2 + R ** 2)
        return cov

P.S.: I have just started working with GPyTorch a few days ago and love it, so thanks a lot for your work!

[Balandat]

Do you have a full repro?

This seems like there could be some numerical issues going on.

First of all, are you using the double dtype? If not, try that.

I don't understand why you're setting r.where(r == 0, torch.as_tensor(1e-20)). This will mess up the scales quite a bit. If you're worried about zero distances between the inputs you should just set those to the proper value - I haven't looked into the paper much but it seems if r is zero then the result should just be R^2.

It may sense to rewrite the covariance like this:

cov = (2 * torch.log(r / R) - 1) * r ** 2 + R ** 2

are your inputs here?