All kappas (and decrease factors) lead to similar median syllable length

[hummuscience]

I am trying out kpms for freely moving mice filmed from above (30fps, similar keypoints to superanimal model, only midline, head, ears, and nose are used). The output is from lightning pose (same format as DLC).

I tried a few kappas as in the example. They all seemed to end up between 5 and 7 frames. I even went a bit higher to see what happens. It seems like the kappa increase does lead to a longer median syllable in the AR model, but then once the SLDS model is initiated, its always goes down (or in some cases up) to around 5-7 frames.

I also wondered if this is due to the decrease factor being too strong. So I tried all combination of 5 kappas and 4 decrease factors

I wonder what is causing this... One thing I was thinking about is the difference between the outputs of lightning pose vs. dlc. The calibration was different than with DLC as LP is over-confident. With the majority of variation happening above 0.9 likelihood. I set the confidence threshold to 0.9. Does that mean that kpms will try to interpolate everything below that?

Something else that I did not try is play with the alpha and gamma parameters. Do you think it is worth it to run some hyperparameter optimization? Any other ideas?

Thanks in advance!

[calebweinreb]

Hey! That's interesting. One thing to note is that interpolation of low confidence keypoints is only used during the ar_only stage. During the full model stage, the original keypoints are used (this is where the slope/intercept outputs of calibration are used). So overconfidence of jittery keypoints could definitely be an issue. Here are two ideas:

1. Use much higher kappas (like 1e10 or 1e15, and maybe try increasing kappa for the full model stage)
2. Use interpolated keypoint data for both stages of modeling. This can be done by setting low confidence values to NaN before you apply the format_data function.

My guess is that alpha and gamma wont impact this much.

[hummuscience]

The calibration part doesn't seem to make much sense in my case since there is little to interpolate. A keypoint is either visible (confidence above 0.95) or not (confidence below 0.01). At least that is what lightning pose outputs. I tried to already go for a model that is less "over confident", but it generally performs really good. Is there a way to turn off the calibration and just set points below a threshold to NaN?

Either way, I tried much higher kappas with the same decrease factors, and it did the trick. Will try a few combinations that converged onto 12-13 frames and see what comes out of that.

[calebweinreb]

Yeah you could definitely set the low-conf points to NaN. I would just do it manually...

for k in coordinates:
    coordinates[k] = np.where(confidences[k][:,:,None] > 0.9, coordinates[k], np.nan)