Add a way to store configurable parameters

[DNedic]

We need a way to store configurable parameters, such as filter coefficients, mixer values, default control modes and more.
These are essentially key-value mappings which don't change very often and don't require high speed or low latency access/modification.

We need to figure out both the storage medium as well as the format and API for storing and reading these.

[DNedic]

For the API and implementation, we could take a look at the new Zephyr Memory Storage subsystem or Zephyr NVS, and regarding the backend we could start simple with an internal flash partition, and possibly switch to an external NOR flash chip.

[lkijanovic]

There are multiple ways to achive the aim of this ticket through Zephyr APIs, I will shortly describe the ones that don't seem to fit our use-cases and then move to list pros and cons of the remaining candidates.

Flash Circular Buffer (FCB) - Not really sure that this would work without extensive wrapper. Also, I'm getting a vibe from Zephyr docs that this might get deprecated at some point in the future.

Zephyr Memory Storage (ZMS) - Seems to be more geared towards non-erasable technologies, even though it can be used with flash memory. Even so, it has bigger metadata overhead than any of the flash alternatives.

File in LittleFS- Probably impractical as we'd need to have bulky logic to serialize and parse the config files. I also see some architectural problems, with most of the other stuff having dependencies to this "config storage" component/library.

So essentially, to me it seems the choice boils down to two alternatives:

Non-Volatile Storage (NVS)

Pros:

• Simple key-value system, low metadata overhead
• Easily mapped to MAVLink parameters, parameter index could be the NVM key

Cons:

• Flash support only, no way to use uSD for storage
• Has to use multiples of flash page size, somewhat problematic for devices with little flash and/or weird heterogenous sectors

Settings API

Pros:

• Supports both flash and uSD, fairly easily extendable to other storage media as well

Cons:

• Slightly more overhead in flash than NVS
• Mapping to MAVLink parameters could require some sort of a LUT

I'm open to discussion on this, but the sole benefit of having option to store to uSD as well is a decisive factor for me. I see some use cases where this comes in handy:

• Support boards with little flash/weird sectors
• Use card storage during e.g. calibration procedures where flash writes could happen often

[DNedic]

I find the argument for interchangeable storage backend convincing, but there are some caveats.

1. NVS and NVM have simpler APIs and we don't really need complex value types the settings feature offers
2. How will we choose which backend to use? With a Kconfig option? In the devicetree? At runtime? What happens when we want to fly without and uSD card? With NVS we would hard require a flash device of at least a certain size and forego the complexity. This would obviously limit the number of boards that can be used.
3. How does the settings feature map to uSD when there is a filesystem already present? How are concurrency problems between them solved?

[lkijanovic]

@DNedic

1. I agree, lower-level APIs are simpler and may suffice for most of the needs we have.
2. If we do opt to go for Settings API, further implementation details are open for discussion. My preference would be to support backend switch only through Kconfig. It would be up to user to make sure their hardware and overlay support the backend they chose.
3. Settings are stored in a file in an existing filesystem and syncing FS operations is responsiblity of the FS layer itself. Browsing through LittleFS implementation seems to confirm this (they use an internal mutex).

[DNedic]

So far, it does seem reasonable to go with the settings API. We can move over chips with either larger internal flash size or more convenient page size layout to use the internal flash with a NVS backend down the road and have the SD card as a fallback.