Motion Detection - frame skipping logistics & anomalies definition

[AndyZhou952]

Thank you so much for this great work!

For Jump Cut and Motion Detection, we mentioned that "We used LPIPS frame-skipping to compute inter-frame semantic similarity, identifying anomalies as cut points and taking the mean as the motion score."

I am curious how exactly the motion scores are calculated, in particular:
(1) How is frame-skipping done? Do we skip the same number of frames for all videos (say 30 frames across all video), or is it also based on fps? For videos with various lengths, do we sample the same amount of frames? Great if we can have these details
(2) Could you help clarify "identifying anomalies as cut points"? Is it that we cut the videos by the cut points? How are "anomalies" defined? From the pipeline it seems that we simply discard the videos with motion scores m > 0.3.

[qqingzheng]

Thank you for asking.

(1) Frame-skipping is simply reading video with a certain frame spacing. This is done to improve I/O efficiency.
(2) Simply put, we define outliers as points that exceed a certain threshold of the variance of lpips values, and we use these outliers to cut the video. Moreover, we have additional strategies to further filter these outliers, which will be presented in our future technical report.

This fig shows what the 'anomalies' look like.