High dimension lengthscales for custom Kernel

[faridf]

We opened up the topic here initially: https://github.com/cornellius-gp/gpytorch/issues/2046#issue-1281317429
I used parameter registering and now the main kernel is aware of such 2d lengthscale but one issue still remains and that is what gpleiss @gpleiss mentioned in https://github.com/cornellius-gp/gpytorch/issues/2046#issuecomment-1281565832

When it comes to testing the sizes of x1 and x2 are different when testing as oppose to when you are training.

Any remedy for this issue?

[faridf]

I am also looking into whether there is a RBFCovariance class that can handle multiple lengthscales,since my length scales have multiple dimensions.
This is the available RBFCovariance class inside gpytoch:
`class RBFCovariance(torch.autograd.Function):
@staticmethod
def forward(ctx, x1, x2, lengthscale, sq_dist_func):
if any(ctx.needs_input_grad[:2]):
raise RuntimeError("RBFCovariance cannot compute gradients with " "respect to x1 and x2")
if lengthscale.size(-1) > 1:
raise ValueError("RBFCovariance cannot handle multiple lengthscales")
needs_grad = any(ctx.needs_input_grad)
x1_ = x1.div(lengthscale)
x2_ = x2.div(lengthscale)
unitless_sq_dist = sq_dist_func(x1_, x2_)
# clone because inplace operations will mess with what's saved for backward
unitless_sq_dist_ = unitless_sq_dist.clone() if needs_grad else unitless_sq_dist
covar_mat = unitless_sq_dist_.div_(-2.0).exp_()
if needs_grad:
d_output_d_input = unitless_sq_dist.mul_(covar_mat).div_(lengthscale)
ctx.save_for_backward(d_output_d_input)
return covar_mat

@staticmethod
def backward(ctx, grad_output):
    d_output_d_input = ctx.saved_tensors[0]
    lengthscale_grad = grad_output * d_output_d_input
    return None, None, lengthscale_grad, None`

[gpleiss]

I am confused why your RBFKernel needs "multiple lengthscales." Are your inputs D-dimensional, and you need D different lengthscale values (one for each dimension)?

It would be helpful if you could describe your data in more detail to avoid further confusion.

[calebnewman]

Hi This was sent to me by accident- pls check email address. C

…

On Tue, 17 Jan 2023 at 02:08, Geoff Pleiss ***@***.***> wrote: I am confused why your RBFKernel needs "multiple lengthscales." Are your inputs D-dimensional, and you need D different lengthscale values (one for each dimension)? It would be helpful if you could describe your data in more detail to avoid further confusion. — Reply to this email directly, view it on GitHub <#2237 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AENOODMHZNBEDMQUF6U5HCTWSX5LJANCNFSM6AAAAAATJEHBLQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>