Improve performance on 4k preview

[geocine]

I have a webcam that supports 4k 30fps. So far the only preview solution I can find in .NET that compares to the Windows Camera app and OBS is the UWPs MediaCapture and MediaElement . Using these controls on WPF works for the most part but the airspace issue hits you and you have to make workarounds when incorporating on your UI.

I love the simplicity of this library and was surprised that it even supports Avalonia. It would be great to have 4k preview support in the future.

[kekyo]

I love the simplicity of this library and was surprised that it even supports Avalonia. It would be great to have 4k preview support in the future.

Thanks for using FlashCap!

Since the release of FlashCap, we have investigated some environments where FlashCap does not yet work, specifically the Android camera system. So we have found a few issues that FlashCap may need to address in the future:

Preview surface support

Many media capture APIs support preview surfaces. This is a feature that allows for extremely fast previews by transferring data directly from the capture device to the GPU, "independent" of the bitmap from the media.

When I first implemented FlashCap, we had a bit of trouble deciding whether to support it or not, but since our first and most important goal was to get bitmap data, we decided not to support it. (FlashCap was largely born out of the necessity of my work.)

I have actually tested FlashCap at about 2K resolution and it captures fine in my environment. However, at resolutions as high as 4K, the frame rate drops drastically when previewing. In the near future (4K resolution is already becoming popular?), it is likely to be possible to use native preview functionality.

I think that there must be some way to support native preview functionality. The challenge is to find a way to handle previews in a unified, transparent, and easy-to-use API across multiple platforms.

Reactive interface

When I first looked into the Android NDK Camera API, I was surprised by its complexity. For good reason: the camera device itself and the attributes it supports (resolution, camera settings, etc.) need to be able to be changed by the user at any time in the Android configuration screen, and the application needs to be able to detect these changes.

For example, a user could open the Android configuration screen and suddenly disable the application's camera permissions or, in some cases, switch the camera to no longer needed. Or they could change the default focus setting of the camera. Thus, the camera API was almost entirely realized through callbacks, and this seemed to be one of the factors complicating things.

To return to our FlashCap, it is actually quite possible that such device changes can occur in Windows and Linux as well, if the camera's USB cable is unplugged or newly connected, etc. Windows and Linux are not as nervous as Android. However, if we want to support a system with a reactive low-level API like Android and possibly iOS in the future, the current FlashCap API may not be able to handle it. Perhaps we need to consider something like the FlashCap V2 API with a reactive interface.

Needs to be...

These are pretty heavy issues, especially since the V2 API is likely to be incompatible with the current API, so I feel this needs to be implemented in a meaningful way.
(This does not mean that they are unwilling to do so. :)