Issues with SysEx in 5.0.2

[LouDou]

Context

Please answer a few questions to help us understand your problem better and guide you to a solution:

• What board are you using ?
  • ESP32
• What IDE are you using ?
  • VSCode with Platform IO
• How are you using MIDI ?
  • Hardware Serial (DIN plugs)
  • USB
  • USB Serial
• Is your problem related to:
  • MIDI Input (reading messages from other devices)
  • MIDI Output (sending messages to other devices)
• How comfortable are you with code ?
  • Complete beginner
  • I've done basic projects
  • I know my way around C/C++
  • Advanced / professional

Describe your project and what you expect to happen:

I'm constructing a project which maps MIDI data to DAC voltages (i.e. MIDI to CV).
I've written a SysEx protocol in order to set/change the MIDI configuration on the ESP32 device, e.g. channel mapping, note ranges and a bunch of other stuff.
Due to the multitude of things happening on the ESP32, in my project I have encapsulated various things into classes, to keep the code organised and things like devices and other IO abstracted away from each other.
(Unfortunately I'm not at the stage where I'm willing to open source this yet, but I'll provide relevant snippets below).

I have a desktop app which constructs the SysEx messages and I'm sending them using loopMIDI and Hairless MIDI to the ESP32 device.

These SysEx messages are (for the time being, until I optimise the config protocol) somewhat "large" at around 180 bytes.
Here is one full message, as constructed on the PC

f0 00 60 00 00 00 00 01 00 00 01 01 00 06 02 01 00 00 03 01 00 14 04 01 00 46 05 01 00 04 06 01 00 00 00 00 01 04 01 00 01 01 02 00 01 00 05 00 01 00 00 02 00 01 01 02 00 03 02 02 00 00 03 02 00 14 04 02 00 46 05 02 00 4a 06 02 00 00 00 00 02 05 01 00 02 01 02 00 02 00 05 00 02 00 00 04 00 00 01 04 00 01 02 04 00 02 03 04 00 00 04 04 00 7f 05 04 00 07 06 04 00 00 00 00 04 06 01 00 04 01 02 00 04 00 05 00 04 00 00 08 00 03 01 08 00 01 02 08 00 03 03 08 00 00 04 08 00 7f 05 08 00 0b 06 08 00 0c 00 00 08 05 01 00 08 04 02 00 08 00 05 00 08 1e f7

There is some SysEx message fragmentation going on, which I can deal with, but there's another more serious issue with this library I think.

Describe your problem (what does not work):

First, well what does work?
Using the callbacks mechanism MIDI.setHandleSystemExclusive(...).
In this case, I always get all the data sent from the PC, although it is fragmented. I am able to buffer the data and remove the {0xF0, 0xF7} continuation marker bytes and extract my config data for the application to use. Great :)

What does not work?
Trying to use callbacks when MIDI is encapsulated in another class... but... you have also provided an API to read the MIDI message data;

void EncapsulatingClass::read()
{
  while (m_midi.read()) // m_midi is your MIDI class instance
  {
    switch (m_midi.getType)
    {
    case midi::MidiType::SystemExclusive:
        processSysEx(m_midi.getSysExArray(), m_midi.getSysExArrayLength());
        break;
    }
  }
}

I was expecting processSysEx to receive the exact same data as the callback function did. However, it does not.
(also as an aside, your array type is different between these two as well, the callback uses byte *array and getSysExArray() returns const byte *array).
In the case of fragmented SysEx messages, it seems to only ever receive the final part of the fragmented message.
In my case I was receiving only an array of length 57 bytes, starting with an 0xF7 and ending with an 0xF7. I never received the first part of the data.

So, there is some key difference between how and when the callbacks are fired, vs. the data available when asking for it via the getX() APIs.

It would be great also if the way you declare the callback interfaces can use <functional>, so that they will accept the various newer ways to reference functors, functions, std::bind and the like. This way, I can bind my EncapsulatingClass methods and set up the callbacks using those.

Alternatively, please tell me I'm doing something wrong? ;)

Regards,
Doug.