Livestream 'lockup' with slow computer

[mungewell]

I am seeing a problem when running on a slow computer (namely a Raspberry Pi), it seems that the image decode is not faster enough and at some point lags so far behind that the camera gives up - closing the connection and leaving the app hanging.

Looking at profiling the code it seems that there is just 1 call to liburl2 which returns a pointer to a 'never ending bowl' from which we read the image frames from in turn.

There is a time stamp in the common header, and mention in the API that:

The client may not display all the JPEG because of the decoding time. In that case, the
client should skip some JPEG images.

I think this is the solution, but can't quite understand (it's late night here :-) how to evaluate the frame rate/time stamp to decide we're running too far behind. The solution will probably fit here:

    if common['payload_type']==1:
       payload = payload_header(data)
+      if payload['timestamp"] : #??? not sure how to eval "much too late"
+         # running too slow, skip displaying this frame
+         read(payload['jpeg_data_size'] + payload['padding_size'])
+         continue
       image_file = io.BytesIO(incoming.read(payload['jpeg_data_size']))
       incoming_image = pygame.image.load(image_file).convert()
       if options.zoom:
          incoming_image = pygame.transform.scale(incoming_image, \
             (infoObject.current_w, infoObject.current_h))
       incoming.read(payload['padding_size'])

[mungewell]

Had a little more of a think about this over my lunch hour;
KAPP9.PDF

Problem with RaspPi (running pygameLiveView) is primarily that the 'receive' is held up by the 'decode/display' resulting in an ever increasing queue of frames in the pipe. There is also the issue of delay in TCP/IP pipeline, but that should be comparatively small.
We could compare rolling average fps (last 'x' frames) for 'capture' vs 'display', and skip if we're ever slower. This probably wouldn't recover any lag after it had been introduced.

[mungewell]

Seems to be working now...
https://www.youtube.com/edit?o=U&video_id=xR3gAfu4Hfw

[krisrok]

man you need to learn how to put links on the interwebs :D

your youtube link links to some 404 github url and the link's text is the non-public url to the video. it should read instead https://www.youtube.com/watch?v=xR3gAfu4Hfw

also on your youtube video the links back to your forked repo is cut off and is another github 404. https://github.com/mungewell/sony_camera_api would be the right one.

link issues aside i'm glad you've managed to get a smooth live stream working as i'm working on something similar :)