MTKView with UIViewRepresentable shows black

[deshan]

Hi
I have a program which render a byte buffer with using Metal. This renderer I attached to the SwiftUI with using a UIViewRepresentable.

struct VideoResource: UIViewRepresentable {
    typealias UIViewType = MTKView

    let metalRendererView: RendererMetal

    func makeUIView(context: Context) -> MTKView {
        let mtkView = MTKView()
        mtkView.device = MTLCreateSystemDefaultDevice()
        mtkView.delegate = metalRendererView
        mtkView.colorPixelFormat = .bgra8Unorm
        return mtkView
    }

    func updateUIView(_ uiView: MTKView, context: Context) {
        uiView.setNeedsDisplay()
    }
}

class RendererMetal: NSObject, MTKViewDelegate {
  func displayFrame(frame: UnsafeMutablePointer<VideoFramePlane>) {
   ...
  }
  func draw(in view: MTKView) {
   .... 
 }
}

When I apply pipify the pip window is black. When it's on SwiftUI it just display fine. Any idea?

Button("Basic Example") { isPresentedThree.toggle() }
                    .pipify(isPresented: $isPresentedThree) {
                        VideoResource()
}

[Sherlouk]

Unfortunately, it's most likely that the ImageRenderer does not support this view.

Per Apple's documentation:

ImageRenderer output only includes views that SwiftUI renders, such as text, images, shapes, and composite views of these types. It does not render views provided by native platform frameworks (AppKit and UIKit) such as web views, media players, and some controls. For these views, ImageRenderer displays a placeholder image, similar to the behavior of drawingGroup(opaque:colorMode:).

If you find a better way to convert a view (either UIKit or SwiftUI) into an image, feel free to open a PR with an opt-in toggle 🙂

[deshan]

Thanks @Sherlouk
Yes it should be the reason. I will make a PR if found a better way.

Since I have Video Data buffer, what would be the best way to render it directly to the AVSampleBufferDisplayLayer?
And my application is two way video communication, in that case where can I modify in the library to use AVPictureInPictureVideoCallViewController?