h264rtpdepacketizer doesn't handle multiple NAL units with same timestamp

[fusionfoxy]

I have a camera generating RTP packets with too high MTU. To fix this, I feed the packets to the h264 rtp depacketizer, and then to the h264 rtp packetizer using a proper MTU. To test this I am using GStreamer:

gst-launch-1.0 videotestsrc ! clockoverlay ! video/x-raw,width=640,height=480 ! videoconvert ! queue !   x264enc tune=zerolatency bitrate=1000 key-int-max=30 ! video/x-h264, profile=constrained-baseline !   rtph264pay pt=96 mtu=1600 ! udpsink host=127.0.0.1 port=6000

This gives me RTP packets where every timestamp contains 2 NAL units with each unit split into 2 packets:

The h264 rtp depacketizer looks for the first packet with NAL unit type FUA and only adds the separator for that. This results in the packetizer merging the 2 NAL units into one which then causes errors in the video player.

If instead of looking for the first FUA packet, I change the depacketizer to check if the NAL unit fragment header isStart() the feed works as expected:

diff --git a/src/h264rtpdepacketizer.cpp b/src/h264rtpdepacketizer.cpp
index f2ddabb4..15126f21 100644
--- a/src/h264rtpdepacketizer.cpp
+++ b/src/h264rtpdepacketizer.cpp
@@ -67,7 +67,7 @@ message_vector H264RtpDepacketizer::buildFrames(message_vector::iterator begin,
                        auto nalUnitFragmentHeader = NalUnitFragmentHeader{
                            std::to_integer<uint8_t>(pkt->at(rtpHeaderSize + sizeof(NalUnitHeader)))};
 
-                       if (nFrags++ == 0) {
+                       if (nalUnitFragmentHeader.isStart()) {
                                addSeparator(accessUnit);
                                accessUnit.emplace_back(
                                    byte(nalUnitHeader.idc() | nalUnitFragmentHeader.unitType()));

[paullouisageneau]

Good catch, thank you for reporting! I push the suggested fix in #1167 (It also ensures a separator is always written at the beginning of the access unit).