Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

360 video support #318

Merged
merged 14 commits into from
Mar 20, 2024
Merged

360 video support #318

merged 14 commits into from
Mar 20, 2024

Conversation

fisheggg
Copy link
Contributor

@fisheggg fisheggg commented Mar 6, 2024

Hi guys! This is the PR for #315. I've implemented most of the functionalities that we discussed, and there are some TODOs left as I will wrote down below.

New functionalities

  • Add Mg360Video class that inherits MgVideo, contains exclusive properties and methods for 360 videos
    • Projection class that is an enum of different types of projections, using the same naming with ffmpeg with a few additions (There are two aliases for equirectangular that are 'e' and 'equirect', but no 'equirectangular' or 'erp' which I think are more common).
    • Mg360Video.Projection property that represents the current projection of the video.
    • Mg360Video.convert_projection() that converts the video into target projection using ffmpeg.
    • The new Mg360Video class can, of course, support all the analysis functions in the original MgVideo class.
  • Add automatic file merging for MgVideo and Mg360Video, by using a list of paths as filename.
    • E.g., mgv = MgVideo(["video_1.mp4", "video_2.mp4"]) will merge the two video files and create the object with the merged file path.
    • Add _utils.merge_videos() that merges multiple video files with ffmpeg.
  • Add detection of regular ipython notebook environment _utils.in_ipynb(). There was a detection for google colab, and now we can detect regular notebooks too. I've also added in _utils.show() that automatically switches to notebook mode if detected.

Fixes

  • Fixed Mg360Video.show(). The original MgVideo.show() uses non embed mode, which does not support large video files. We change the default of Mg360Video.show() to embed mode, and that does not affect the original MgVideo.show().
  • Fixed an error that MgVideo.show() does not support video files with non-lower case extension name, e.g. "video.MP4". That happens for some cameras such as the Garmin 360 VIRB which saves files in capital extension name.

TODOs

  • Support for .360 files from the GoPro MAX. It is a customized file format that requires extra stream processing and stiching. We've discussed the format in our coming paper, but some extra effort is needed to support the format in MGT.
  • Add configurations for other 360 cameras (insta360 X3, Garmin VIRB 360, ricoh theta s, etc..). There are extra information needed for the projection conversion, such as the orientation of the fisheyes, the field of view angles of the cameras, etc. I need some time to manually figure out their configurations and add into _360Video.CAMERA.

Cheers! 🍻
Arthur

@joachimpoutaraud
Copy link
Contributor

joachimpoutaraud commented Mar 20, 2024

Very nice work @fisheggg! I have reviewed your new functionalities on 3 different OS and Python versions, and all checks have passed smoothly. Therefore, I have merged your pull request.

If you have time, could you provide some code snippets for using all your new functionalities in the Wiki page? That would be amazing!

@joachimpoutaraud joachimpoutaraud merged commit 95f149c into fourMs:master Mar 20, 2024
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants