How to save homography matrix (for video)

[HugoDeCozar]

Hello!

I am stitching frames from two videos (left and right camera) to make a unique video. Stitching individual frames is fine, but when the resulting frames are combined in the final video, the result is very messy. This is because the images will warp a lot from one frame to the other.

Ill post two frames as an example

I guess the solution is to get the homography matrix from stitching the first frame, and using it in the other frames. I got that idea from this tutorial: https://pyimagesearch.com/2016/01/25/real-time-panorama-and-image-stitching-with-opencv/

How would you do that?

[lukasalexanderweber]

You could try something like this (not tested!):

from stitching import Stitcher
from stitching.images import Images

class VideoStitcher(Stitcher):

    def initialize_stitcher(self, **kwargs):
        super().initialize_stitcher(kwargs)
        self.cameras = None
        self.cameras_registered = False
		
    def stitch(self, images, feature_masks=[]):
        self.images = Images.of(
            images, self.medium_megapix, self.low_megapix, self.final_megapix
        )

        if not self.cameras_registered:
            imgs = self.resize_medium_resolution()
            features = self.find_features(imgs, feature_masks)
            matches = self.match_features(features)
            imgs, features, matches = self.subset(imgs, features, matches)
            cameras = self.estimate_camera_parameters(features, matches)
            cameras = self.refine_camera_parameters(features, matches, cameras)
            cameras = self.perform_wave_correction(cameras)
            self.estimate_scale(cameras)
            self.cameras = cameras
            self.cameras_registered = True

        imgs = self.resize_low_resolution(imgs)
        imgs, masks, corners, sizes = self.warp_low_resolution(imgs, self.cameras)
        self.prepare_cropper(imgs, masks, corners, sizes)
        imgs, masks, corners, sizes = self.crop_low_resolution(
            imgs, masks, corners, sizes
        )
        self.estimate_exposure_errors(corners, imgs, masks)
        seam_masks = self.find_seam_masks(imgs, corners, masks)

        imgs = self.resize_final_resolution()
        imgs, masks, corners, sizes = self.warp_final_resolution(imgs, self.cameras)
        imgs, masks, corners, sizes = self.crop_final_resolution(
            imgs, masks, corners, sizes
        )
        self.set_masks(masks)
        imgs = self.compensate_exposure_errors(corners, imgs)
        seam_masks = self.resize_seam_masks(seam_masks)

        self.initialize_composition(corners, sizes)
        self.blend_images(imgs, seam_masks, corners)
        return self.create_final_panorama()

So basically you get the cameras with your first image set and afterwards use the obtained cameras for all other frames

[earthscience2]

The three images were stitched using the corresponding code. Although the camera variables were stitched while fixed, a small part of the screen continued to change in the created image. Is it normal?

[lukasalexanderweber]

No

[HugoDeCozar]

Works great! Thanks a lot

The only change i had to make is placing imgs = self.resize_medium_resolution() ABOVE if not self.cameras_registered.

[lukasalexanderweber]

Great! I think changing

imgs = self.resize_low_resolution(imgs)
to
imgs = self.resize_low_resolution()

should also work

[HugoDeCozar]

Another thing: my frames now take 0.2s each to stitch. Is there a way to make the process faster? I already have try_use_gpu=True.

[lukasalexanderweber]

Great! I think changing

imgs = self.resize_low_resolution(imgs) to imgs = self.resize_low_resolution()

should also work

I think this is faster, since the images don't need to be resized twice.

If you don't need the full resolution you can reduce it by setting final_megapix.

You maybe don't need exposure compensation, seam finding or blending, depending on your requirements on the result
compensator="no", blender_type="no", finder="no"

[HugoDeCozar]

thanks! will try

[BALLABLLU]

Can I use a GPU to increase the frame rate? For now, I want to use a while loop and a buffer to achieve real-time panorama stitching.

[lukasalexanderweber]

I'm by far not an expert in GPU acceleration, and many people already asked in the forum If its possible with stitching. I was never able to test CUDA support since I don't have NVIDIA hardware.

[BALLABLLU]

Thank you for answering me Lukas Weber ***@***.***>于2023年10月9日 周一上午1:26写道：

…

I'm by far not an expert in GPU acceleration, and many people already asked in the forum If its possible with stitching. I was never able to test CUDA support since I don't have NVIDIA hardware. — Reply to this email directly, view it on GitHub <#148 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AT44RPKBPVTQWFWVGXG53MTX6ODJHAVCNFSM6AAAAAA5LF4VQGVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TEMRWHAYDK> . You are receiving this because you commented.Message ID: ***@***.***>

[deepConnectionism]

I also encountered this problem. How can I stitch images from 8 1080P cameras in real time? Need to achieve 30 fps

[iiiceice]

Hello!

I am stitching frames from two videos (left and right camera) to make a unique video. Stitching individual frames is fine, but when the resulting frames are combined in the final video, the result is very messy. This is because the images will warp a lot from one frame to the other.

Ill post two frames as an example

I guess the solution is to get the homography matrix from stitching the first frame, and using it in the other frames. I got that idea from this tutorial: https://pyimagesearch.com/2016/01/25/real-time-panorama-and-image-stitching-with-opencv/

How would you do that?

can you share that original code? cause i found that link is to subscribe, and that costs much, i can't afford that.PLEASE!