Check for out of order packets

[leskerr]

UDP packets are not guaranteed to arrive in order. Although out of order packets seem to happen very rarely, I have evidence that this may have occurred while I was monitoring wire 105 just now. The string MPH, decoded as MPH * H * ,. Here are the packets that were received by MKOB:

{"ts": 1608933090574, "w": 113, "s": "Beaufort Scale, 30 wpm, AC", "o": 2, "c": [-306, 120, -40, 120]}
{"ts": 1608933091088, "w": 113, "s": "Beaufort Scale, 30 wpm, AC", "o": 2, "c": [-153, 40, -40, 40, -40, 40, -40, 40, -40, 40]}
{"ts": 1608933091521, "w": 113, "s": "Beaufort Scale, 30 wpm, AC", "o": 2, "c": [-153, 40, -40, 40, -40, 40, -40, 40]}
{"ts": 1608933091826, "w": 113, "s": "Beaufort Scale, 30 wpm, AC", "o": 2, "c": [-32767, 40, -40, 40, -40, 40, -40, 40]}
{"ts": 1608933092269, "w": 113, "s": "Beaufort Scale, 30 wpm, AC", "o": 2, "c": [-32767, 40, -40, 120, -40, 40, -40, 120]}

The 'H' code sequence appears twice. The -32767 elements indicate what pykob/internet.py perceived as gaps in the sequence numbers of the received packets. MKOB clients always send each packet twice, with the same sequence number, to guard against single missing packets. In this case, it appears the first packet for the comma arrived before the second packet of the 'H'.

[Thank you again, @AESilky, for the incredibly useful Record/Play feature. Having it enabled by default allowed me to analyze this unexpected event after the fact.]

It would also be useful to log single missing packets. Although single missing packets don't interfere with the operation of the program, knowing that they're happening can provide a warning that the internet connection is less than solid.

[jchausler]

<!-- DIV {margin:0px;} -->Hi Les,I have seen such patterns on occasion as well.  Two other odd occurrences I see occasionally are duplicated letters and missing letters with no asterisk.  I've seen both on the weather wire.  Just FYI.  As you say, they are rare.  73, Chris-----Original Message----- From: Les Kerr Sent: Dec 26, 2020 11:52 AM To: MorseKOB/PyKOB Cc: Subscribed Subject: [MorseKOB/PyKOB] Check for out of order packets (#208) UDP packets are not guaranteed to arrive in order. Although out of order packets seem to happen very rarely, I have evidence that this may have occurred while I was monitoring wire 105 just now. The string MPH, decoded as MPH * H * ,. Here are the packets that were received by MKOB: {"ts": 1608933090574, "w": 113, "s": "Beaufort Scale, 30 wpm, AC", "o": 2, "c": [-306, 120, -40, 120]} {"ts": 1608933091088, "w": 113, "s": "Beaufort Scale, 30 wpm, AC", "o": 2, "c": [-153, 40, -40, 40, -40, 40, -40, 40, -40, 40]} {"ts": 1608933091521, "w": 113, "s": "Beaufort Scale, 30 wpm, AC", "o": 2, "c": [-153, 40, -40, 40, -40, 40, -40, 40]} {"ts": 1608933091826, "w": 113, "s": "Beaufort Scale, 30 wpm, AC", "o": 2, "c": [-32767, 40, -40, 40, -40, 40, -40, 40]} {"ts": 1608933092269, "w": 113, "s": "Beaufort Scale, 30 wpm, AC", "o": 2, "c": [-32767, 40, -40, 120, -40, 40, -40, 120]} The 'H' code sequence appears twice. The -32767 elements indicate what pykob/internet.py perceived as gaps in the sequence numbers of the received packets. MKOB clients always send each packet twice, with the same sequence number, to guard against single missing packets. In this case, it appears the first packet for the comma arrived before the second packet of the 'H'. [Thank you again, @AESilky, for the incredibly useful Record/Play feature. Having it enabled by default allowed me to analyze this unexpected event after the fact.] It would also be useful to log single missing packets. Although single missing packets don't interfere with the operation of the program, knowing that they're happening can provide a warning that the internet connection is less than solid. —You are receiving this because you are subscribed to this thread.Reply to this email directly, view it on GitHub, or unsubscribe.

[AESilky]

@leskerr, I'm glad it has proven helpful once again. I think that it is helpful to have it enabled, but I would like to make saving the file an option. That way, the folder doesn't fill up with a bunch of session recordings that you don't care about (and many people might not even know about).

[leskerr]

@leskerr, I'm glad it has proven helpful once again. I think that it is helpful to have it enabled, but I would like to make saving the file an option. That way, the folder doesn't fill up with a bunch of session recordings that you don't care about (and many people might not even know about).

Yes, I agree. Saving the JSON file should be an option. (@pwdirks, take note for the options dialog.)

[AESilky]

@leskerr, @pwdirks, It might not need to be an 'option' in the dialog. I think it could be in the 'File' menu for now ('Save current session recording') and in the 'Record/Play GUI' once we have that implemented. That GUI section is what we have been talking about that would have the 'typical' VCR type controls for record and playback.

I could generate the name as I do now, but actually save to a temp file. Only if they say to save would I save the file to the generated name (by default) or to a name they choose. On exit, the temp file would be deleted.

I'll create an issue for that.

[pwdirks]

Hi guys, I like the idea of starting a (by default) “disposable” recording locally that you can “Save” using an option from the file menu locally. Actually if you really want to get fancy you could allocate a buffer in memory (or on disk, I suppose) of all the data received so far that you can scrub through kind of like my current car radio does: while you’re listening to a station it records (up to some limit, like 20 minutes) and you can hit “rewind” or “pause” anytime and resume behind real-time. If you’ve hit rewind you can also skip back ahead until you’re listening in real time again. “Rewind / Pause / Skip” are controls you could have in the main panel. “Save” could be in the “File” menu as Ed suggested, and mean saving the contents of the record buffer. I do think it’d be useful to have an option that controls this process unless we limit the size of the in-memory buffer to something harmless. For a museum display you might not want to do a recording. OTOH a configurable limit on the size of the recording buffer might be equally workable, and then even novice users in a museum might have a rewind/skip function? 73, N6WPD -Patrick.

…

On Dec 28, 2020, at 1:25 AM, Ed Silky ***@***.***> wrote: It might not need to be an 'option' in the dialog. I think it could be in the 'File' menu for now ('Save current session recording') and in the 'Record/Play GUI' once we have that implemented. That GUI section is what we have been talking about that would have the 'typical' VCR type controls for record and playback. I could generate the name as I do now, but actually save to a temp file. Only if they say to save would I save the file to the generated name (by default) or to a name they choose. On exit, the temp file would be deleted. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#208 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALTAYKZ5TS5ZT76BDTH4I7LSXBFKDANCNFSM4VKBPHYA>.

[leskerr]

These are good ideas, Patrick. Kind of like a black box flight recorder, always running in the background but available to provide recent history if and when it's needed.

~Les

[AESilky]

The current 'Play' functionality provides pause, skip back n seconds, skip forward n seconds, go to the beginning of the current sender, go to the end of the current sender, while playing a recording. It would be possible to play the current recording while it is being recorded - which is basically what the DVR or Digital Radio is doing when it allows you to pause, move about, and resume 'live' broadcasts.

There are a few things that I think are higher priority - a reliable install, a Raspberry Pi mode, multiple configurations - but most of what is needed is already there.

Currently the code activity is always being recorded, so it is already there. It is just that it is currently being saved to a permanent file as it is being recorded. My first change would be to create a temporary file and write the activity to it and only make it permanent if the user chooses to save it.

[pwdirks]

On Dec 28, 2020, at 11:27 PM, Ed Silky ***@***.***> wrote: The current 'Play' functionality provides pause, skip back n seconds, skip forward n seconds, go to the beginning of the current sender, go to the end of the current sender, while playing a recording. It would be possible to play the current recording while it is being recorded - which is basically what the DVR or Digital Radio is doing when it allows you to pause, move about, and resume 'live' broadcasts. There are a few things that I think are higher priority - a reliable install, a Raspberry Pi mode, multiple configurations - but most of what is needed is already there. Currently the code activity is always being recorded, so it is there. It is just that it is currently being saved to a permenat file.

I wonder if it’d be simpler if the default recording was buffered in memory only? To be discarded on exit, unless explicitly saved to disk first? That might avoid littering the filesystem with recording files (unless they’re all given the same default name unless saved)?

…

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#208 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALTAYKYHOUMDJS22JGBVNFDSXGAEZANCNFSM4VKBPHYA>.