[NTS-1 mkII / NTS-3] The behavior of the frac parameter in header.c is incorrect

[boochow]

Describe the bug

The number shown on display differs from what the API documents say. Therefore, the behavior is also different from that of the drumlogue.

When fracmode = 0, the display shows incorrect numbers when frac is greater than 3.
When fracmode = 1, the display shows "(parameter value) / (10 ^ frac)," which is inconsistent with the API description.

for example:
frac=4 and fracmode = 0:

valule	0	1	2	3	4	5
display	0	0.125	0.125	0.250	0.250	0.375
should be	0	0.0625	0.125	0.1875	0.250	0.3125

frac=3 and fracmode = 1:

valule	0	1	2	3	4	5
display	0	0.001	0.002	0.003	0.004	0.005
should be	0	0.033	0.067	0.100	0.133	0.166

To Reproduce
I tested the following parameter description in header.c on NTS-3.

      {0, 5, 0, 0, k_unit_param_type_none, 1, 0, 0, {"PARAM1"}},
      {0, 5, 0, 0, k_unit_param_type_none, 2, 0, 0, {"PARAM2"}},
      {0, 5, 0, 0, k_unit_param_type_none, 3, 0, 0, {"PARAM3"}},
      {0, 5, 0, 0, k_unit_param_type_none, 4, 0, 0, {"PARAM4"}},
      {0, 5, 0, 0, k_unit_param_type_none, 1, 1, 0, {"PARAM5"}},
      {0, 5, 0, 0, k_unit_param_type_none, 2, 1, 0, {"PARAM6"}},
      {0, 5, 0, 0, k_unit_param_type_none, 3, 1, 0, {"PARAM7"}},
      {0, 5, 0, 0, k_unit_param_type_none, 4, 1, 0, {"PARAM8"}}

Expected behavior

The API document says that the fracmode parameter is:

When set to 0, values will be assumed to be fixed point with the lower frac bits representing the fractional part. When set to 1, values will be assumed to include a fractional part that is multiplied by 10 times frac number of decimals, allowing for base 10 fractional increment/decrements.

In the case of fracmode = 0:
Show value/ (2^frac) on display. (the integer value for setParameter() is displayed_number * (2^frac)).

In the case of fracmode = 1:
Show value/ (10 * frac) on display. (the integer value for setParameter() is displayed_number * 10 * frac).

[dukesrg]

Confirmed. Fractional mode display is bugged. Even not knowing the exact value, the dieplay sequence might be like:
20.25
20.50
20.75
20.00
21.25
i.e. the whole and fractional parts are not in sync at all.

[dukesrg]

Performed some more tests:

• NTS-3 does display only two edge values for the params with min or max over 15-bit range (with allo below implication to the actual displayed value)
• The fractional part (at least for wide span paramater range) always shown as 2 highest fractional part decimal digits. i.e. having parameter with higher frac value can render the displayed value imprecise
• The integer part (at least for wide span paramater range) is always trimmed to 3 lowest integer part decimal digits (even if the fractional part is not displayed)

There is also some issues in KORG Kontrol Editor 2.4 regarding fractional parameters:

• Values and value ranges for fracmode=1 and frac>0 are calculated as x/(10*frac), while NTS-3 itself shows as x/(10^frac) like @boochow described above (IMO the latter was intended, but lost in implementation years ago :()
• Value ranges for fracmode=1 and frac>0 are calculated twice as much on form open, while altering the raw min/max values, e.g. with +/- buttons or entering the explicit value, recalculates the corresponding range value appropriately.