move WF min / WF max sliders to "more" menu?

[rikvanriel]

Would it make sense to move the WF max & WF min sliders from the normal menu to the "more" menu, so these sliders are not always taking up screen space?

This make make it a little easier to use kiwisdr on a smaller screen, and will leave a slightly larger amount of the waterfall visible.

If people think this is a good idea, I am willing to implement that change in the code and simply submit it as a pull request for John to integrate. However, before doing that kind of thing I would like people's opinions :)

[peejii73]

For a small screen that certainly is a good idea .. on the other hand, before we get nice automation to these (especially WF min!) it should be rather easily accessible. At least I find myself fiddling with it quite often..

[rikvanriel]

Needing to fiddle with them often may indicate a problem with the waterfall code. It looks like large power changes at the low end of the power scale create only a small color difference, while large power changes at the high end of the power spectrum generate a large color difference.

On the other hand, seeing the difference between a strong signal and a really strong signal is not all that important, while seeing the difference between the noise floor and a weak signal is.

I am not sure if we can fix this by changing the spectrum generated by mkcolormap(), or whether waterfall_color_index() needs to select the color on a "stronger logarithmic" scale. At first blush these two answers appear the same, but the 8 bit resolution in the color map may mean that one of the two works better than the other.

[peejii73]

Of course the environment also dictates the need for adjustments - in my case my wire antenna is not very sensitive on VLF, but fortunately S/N is rather good, so I can pick up signals easily. To make them visible, I have to move the WF min slider to rather low value, usually somewhere around -130 dBm. Again, moving to 80 meter band that setting washes the waterfall rather white: there -120 .. -110 is much better value.

Knowing not (yet) so much of the under-the-hood workings of this wonderful little radio, I wonder could it be done this way: calculate an average of one line of visible WF data, then adjust WF min so that minimum values produce something barely visible. Kind of AGC, with appropriate filtering :)

[jks-prv]

I change the "WF min" value all the time. Maybe just WF max should go to the extended panel? Maybe that's too confusing. The volume slider could go there too. I never change that. Always use my laptop volume control I'm used to. There is a open request for being able to hide the controls panel like the others.

Yes, I don't even remember where the colormap was pulled from. CuteSDR maybe. Professional spectrum analysis tools, and even WSPR I think, allow all kinds of colormap color selections, but also linearity adjusting. I personally always like to adjust WF min so the noise floor shows some blue color rather than black. The green hues cover too much dynamic range I think.

OpenWebRX has a feature now to "auto-adjust" the waterfall colors. I don't know if it does this by measuring the noise floor. Probably. For us this would be problematic at low zoom factors. But quite useful when zoomed all the way in.

[rikvanriel]

I hope to sit down with a piece of paper and a drawing program over the next few days, and come up with a color scheme that changes about the same amount (to my eye) for every 10dB or so change in signal strength, assuming the waterfall will usually span about 80-150dB in signal strength.

The bottom 30-40dB or so can probably just be shades of blue (going into low green), as long as the color progression is such that signals that are 5-10dB above the noise floor are always visible anywhere inside that range. The yellow, orange, and red ranges can be a little compressed from what they are today, because the difference between a very strong and a really very strong signal are simply not as important.

I will post a patch for testing once I think I have something suitable, so other people can play around with it too, and we can figure out how well it works (or breaks) before doing anything else with the code, like committing it...

[jks-prv]

Okay, thanks. The full color spectrum in practice seems to be most useful for making loud MW/SW AM BC stations look nice. I looked at the WebSDR colormap once. Even though it's only two color (purple -> white) it has really good dynamic range properties. I never understood the colormap construction algorithm though.

[rikvanriel]

I think I understand the technical requirements of the color map. The total dynamic range depicted in 256 colors is between 150 and 60dB. Lets arrange these colors in an image that is 256 pixels along one axis. An SSB signal takes 6dB S/N to be understandable. Lets round this down to 5dB.

The above means every 5dB dynamic range is represented by somewhere between 7 and 21 pixels in the color map image. We want every understandable SSB signal to be clearly visible in the spectrum, which means every 7 pixels the color map needs to have clearly changed colors. Anywhere along the total range of colors in the color map. starting at any color, 7 colors further needs to have a clearly visible difference.

That would give us a full dynamic range across the entire spectrum, at essentially any waterfall dynamic range. With the more typically used narrower ranges (~100dB across the dynamic range) that would give us a more comfortable ~12 pixels color difference in the palette for a barely understandable SSB signal.

Now to come up with a color palette that fits these requirements...

[rikvanriel]

This is what the original palette looks like. As you can see, there are several ranges of ~20 pixels with nearly indistinguishable colors. This is not good for distinguishing signals right above the noise levels, and explains why people find themselves needing to adjust the waterfall minimum frequently.

It also very clearly illustrates John's point that green covers way too much dynamic range.

[rikvanriel]

It looks like mapping the values to the color palette in a different way, to "stretch out" the bits of the color palette covered by low power levels, and to compress the bits of the color palette covered by the high power levels, does the trick here. I can suddenly see the signals that are near the noise floor (the spectrum display suggests ~5dB over the noise) quite clearly, and they continue to show while changing the "WF min" slider around in about a 30-40dB range.

Should I file a pull request with this change, or are there other waterfall issues people would like to see addressed first?

--- a/web/openwebrx/openwebrx.js
+++ b/web/openwebrx/openwebrx.js
@@ -2461,7 +2461,7 @@ function waterfall_color_index(db_value)
        if (db_value < mindb) db_value = mindb;
        if (db_value > maxdb) db_value = maxdb;
        var relative_value = db_value - mindb;
-       var value_percent = relative_value/full_scale;
+       var value_percent = Math.sqrt(relative_value/full_scale);

        var i = value_percent*255;
        i = Math.round(i);

[rikvanriel]

Running with this change for the past half hour or so, I can now see SSB signals on the waterfall that are too weak to listen to. This is a distinct improvement from before, when I was able to listen to SSB signals that were invisible on the waterfall.

[clumens]

I've just been playing around with this for a few minutes here and in my unscientific opinion, signals appear a bit sharper. That is, they stick out from the noise a little bit better, probably due to having more well-defined edges to them.

[G8JNJ]

Hi,

I also find that I'm using the waterfall sliders a lot, especially the Minimum as the noise floor varies quite a bit from the LF bands to the HF end of the spectrum.

I think modifying the colour scheme to enhance the range at lower signal levels is a good idea, as once a signal is well above a certain S/N ratio it doesn't really matter how much stronger it is. If you do want to know, selecting the spectrum display will quickly tell you where all the strong signals are at a glance.

However it is useful to be able to pick out the very weak signals from the noise and the modified colour scheme should definitely help with this.

If you want to move the Max Min sliders, why not have them down the vertical edge of the waterfall as in SDR Sharp ?

Also please don't hide the Volume control, I use it all the time as I have a sound bar on my monitor and it's not easy to quickly adjust the volume between different applications.

From my perspective the best solution to improve waterfall viability, would be to have the Grey control panel slide off to the right hand side and have all the controls in the same panel including the ones currently in the 'More' pane.

Regards,

Martin - G8JNJ

[rikvanriel]

The patch above (also in my pull request) expands the color range at lower signal levels, and squeezes it at higher signal levels. The WF min & max settings are no longer very critical to adjust at all.

An alternative implementation might squeeze the color palette instead, but then we would have only 8 hues of blue in the spectrum, which might again result in not enough color difference at the lower signal levels. Doing the squeezing in the function that maps signal levels to the color palette seems more likely to accomplish what you describe above.

Martin, could I convince you to try out my code change above, and see if it helps your situation like it did for clumens above?

[G8JNJ]

Hi Rik,

Sure I'd like to try it.

I'm not great with Linux so could you tell me which files I need to swap, or better still what command string I should use to pull it ?

Regards,

Martin - G8JNJ

[rikvanriel]

We really should have a wiki somewhere with instructions like this, but here it goes:

1. ssh into the kiwisdr
2. cd to /root/Beagle_SDR_GPS
3. edit web/openwebrx/openwebrx.js making the change from the patch above
4. make install
5. service kiwid restart

[G8JNJ]

Hmmm,

Not sure if I've edited the correct section ?

function waterfall_color_index(db_value)
{
// convert to negative-only signed dBm (i.e. -256 to -1 dBm)
// done this way because -127 is the smallest value in an int8 which isn't enough
db_value = -(255 - db_value);

if (db_value < mindb) db_value = mindb;
if (db_value > maxdb) db_value = maxdb;
var relative_value = db_value - mindb;
var value_percent = Math.sqrt(relative_value/full_scale);

var i = value_percent*255;
i = Math.round(i);
if (i<0) i=0;
if (i>255) i=255;
return i;

}

Anyway whatever I've done the colouring has changed, but the waterfall now looks very 'grainy' and no amount of adjustment of the Max / Min sliders seems to make it look any better without loosing the full colour range.

I can see the weaker signals better but the gaps between the stronger stations which were visible when fully zoomed out are now filled with coloured 'specks'

I'm not sure if it's preferable to the previous display, maybe a different log law is required ?

Take a look and see what you think .

http://southwest.ddns.net:8073/

Regards,

Martin - G8JNJ

[rikvanriel]

Indeed, it looks like your noise floor on lower HF and MF is significantly higher than mine, around -80dBm for you, where it is around -110dBm for me.

In my situation, I have found that setting the "WF min" slider to anywhere between 10-40dB below the noise floor seems to give useful waterfall colors. I estimate the noise floor from looking at the spectrum display.

On your system, zooming all the way out, the noise floor from 10-30 MHz appears to be about -90dBm, while (on zoom level 0) the noise floor below 10MHz appears to be closer to -70dBm. However, zooming in to smaller zoom levels, the noise level even between MW and 160m appears to also be around -90dBm.

Setting "WF min" anywhere below -110dBm and -140dBm seems to give reasonable results all over the spectrum for your setup, with signals that are as little as 5dB over the noise floor (according to the spectrum display) having a color that is distinguishable from the background noise.

However, even on your system (in Europe, with blowtorch AM broadcasters), the top 15% or so of the color range is totally unused, and the low range could indeed be stretched out even further.

I guess the square root is better than it is today, but we may want an even more aggressive method. I'll look into it later, hopefully tonight or tomorrow, worst case only in a week and a half.

[rikvanriel]

Thinking about the stuff above some more:

1. with 100dB space between WF min and WF max, a 20dB difference will always be 20% of the color spectrum at least, and will always result in "coloured specks" of one hue or another
2. the square root code is probably fine, it works decently on the low end of the power scale, but
3. the top end of the power scale is barely used, so I should consider narrowing the color bands at the bottom of the spectrum, and stretching out the red->pink area to cover as much as 1/4 to 1/3 of the spectrum, from today's 1/6th

This is something I should be able to do relatively easily by adjusting the numbers in mkcolormap a little bit.

[jks-prv]

Sorry guys. I'm just getting hammered with other issues at the moment.

Yes, the "quick start" web page is getting out-of-control and needs to get turned into a proper wiki (github possibly) with a section on compilation. Some shortcuts for rebuilding if you are logged in as root on the Beagle:
'cdp' change to build directory
'mst' (make stop) stops the background server so your compile runs faster
[make some edits]
'm' to make
'k' to try the server in foreground mode to see more debugging messages
'mi' (make install) install the new version as the background server when you're happy with changes
You don't have to do this of course if you're just experimenting. You could just restart the unmodified background version.
'msa' (make start) or 'ku' (kiwi up) restarts the background server
Additionally:
The 'kd' (kiwi down) command keeps the background server running, but makes it display a "down for development" message when people attempt to connect. I don't do this much anymore since there are so many alternatives to listen to on sdr.hu now.

[G8JNJ]

Hi Rik,

Which is your SDR so that I can take a look ?

I have got a bit of noise around 1MHz to 6MHz when my Heating system and Solar array is running during the day, but it should be better at night, although there is still a bit of VDSL noise from overhead phone lines. I also get lots of lightning pulses which cover the 1MHz to 10MHz range, which I don't see on other folks SDR's. My antenna gain falls rapidly below 1MHz, and I suspect that if this wasn't the case it would be really problematic on the lower bands.

Also I have added some 10dB deep notch filters to take out the strongest BC stations on 900KHz, 6MHz and 9MHz. This was in order to reduce the possibility of overload during the night and also to help flatten out he overall dynamic range.

I've just removed the filters so you can now see the full dynamic range.

Regards,

Martin - G8JNJ

[rikvanriel]

My SDR is http://kiwisdr.surriel.com/

Currently it is attached to the antenna in parallel with a Kenwood TS590 transceiver, which automatically switches between the various JT65/JT9 and PSK31 frequencies, resulting in elevated noise levels when some band filters on the TS590 are switched in. Every few minutes you can see band changes happen.

Eventually it should be on antennas all by itself, through a 6x2 switch (and later an additional 3x2 switch for the ladder line switched antennas, and a 2x2 switch to switch between those switches...)

[G8JNJ]

OK,

Just quickly looked at all the KiWI SDR's on sdr.hu.

The vast majority of them don't have any signals exceeding -50dB, some are so bad I don't think they have an antenna connected at all :-(

A few have some MF BC stations that peak up to about -40dB or so but the higher frequencies are dead.

Apart from my own, during my quick survey I could only find three that had anything like a reasonable distribution of signals across the whole spectrum.

http://mediaexplorer.ddns.net:8073
http://jerseyshoresdr.hopto.org:8073
http://n4tvc.zapto.org:8073/

I see on yours that you don't suffer from the strong SW broadcast stations on 6, 9 11 & 13MHz, but I also notice that your max level is about -30dB (10dB down from mine) and your HF 20 - 30MHz noise floor is about 10dB down on mine. So overall I think my raised noise floor is simply as a result of 10dB more antenna gain at my end.

I can put a 10dB or 20dB pad in line if you wish to perform more tests.

Regards,

Martin - G8JNJ

[G8JNJ]

Hi Rik,

Your waterfall doesn't look anything like as 'grainy' as mine when I adjust the noise floors to compensate for the difference in levels.

Can you check in my previous mail that I have modified the correct part of the file ?

Regards,

Martin - G8JNJ

[rikvanriel]

You appear to have done the modification right.

You just live much closer to very strong HF broadcasters, and your vertical antenna picks up MW broadcasters a lot stronger than the (horizontal 20-10m) lazy H antenna currently connected to my kiwisdr. MW broadcasters in Europe also have much, much stronger signals than MW broadcasters in the US. The strongest MW signal on my waterfall is a 1kW station about 1 mile away.

[icm7216]

Hi,
This patch. I think that affects the spectrum value. Attached image is comparing the signal level of the local broadcasting station. In this patch, It looks like the signal level of the intermediate area is lifted.

Please see attached image:

• original code use (linear version)
  
• this patch use (sqrt version)
  

[rikvanriel]

Indeed. Expanding the bottom and compressing the top has the effect of lifting the area in-between. I don't think there is any way of expanding the bottom that does not lift the area above it.

The point of this change (and a potential further change to the spectrum) is to ensure that any signal that is ~5dB or so over the noise is visible in the spectrum, and do that without being overly sensitive to the WF min and WF max settings.

[rikvanriel]

I have looked at some slight tweaks to the color palette, which increase the red/pink fade over a much larger part of the dynamic range (very strong through extremely strong signals), and reduce the dynamic range represented by green, cyan, and blue.

Now I can see SSB signals that are so far in the noise I am unable to understand them.

This is the change to mkcolormap(). I would like any input by others on whether or not this works right for them (yes, you will see even more colors now), before I create a pull request for John.

--- a/web/openwebrx/openwebrx.js
+++ b/web/openwebrx/openwebrx.js
@@ -2430,18 +2430,18 @@ function mkcolormap()
                                r = 255; g = 255; b = i-256*2;
                        }
                } else {                // CuteSDR
-                       if (i<43) {
-                               r = 0; g = 0; b = i*255/43;
-                       } else if (i<87) {
-                               r = 0; g = (i-43)*255/43; b = 255;
-                       } else if (i<120) {
-                               r = 0; g = 255; b = 255-(i-87)*255/32;
-                       } else if (i<154) {
-                               r = (i-120)*255/33; g = 255; b = 0;
-                       } else if (i<217) {
-                               r = 255; g = 255-(i-154)*255/62; b = 0;
+                       if (i < 32) {
+                               r = 0; g = 0; b = i*255/31;
+                       } else if (i < 72) {
+                               r = 0; g = (i-32)*255/39; b = 255;
+                       } else if (i < 96) {
+                               r = 0; g = 255; b = 255-(i-72)*255/23;
+                       } else if (i < 116) {
+                               r = (i-96)*255/19; g = 255; b = 0;
+                       } else if (i < 174) {
+                               r = 255; g = 255-(i-116)*255/57; b = 0;
                        } else {
-                               r = 255; g = 0; b = (i-217)*128/38;
+                               r = 255; g = 0; b = (i-174)*128/80;
                        }
                }

... looking at it some more, I accidentally narrowed the range taken up by yellow. That is of course not my intention, and I will make another version where that is corrected. Yellow is quite a useful color on the waterfall, and I would like to keep a nice range of yellow available.

[rikvanriel]

Moving red from 174 to 184 seems to do the trick. That changes three numbers in the last few lines of the diff:

+           } else if (i < 116) {
+               r = (i-96)*255/19; g = 255; b = 0;
+           } else if (i < 184) {
+               r = 255; g = 255-(i-116)*255/67; b = 0;
            } else {
-               r = 255; g = 0; b = (i-217)*128/38;
+               r = 255; g = 0; b = (i-184)*128/70;
            }
        }

[rikvanriel]

Playing around with the above some more, I see a few things:

1. There is about a 50dB usable range for WF min where a signal that is 5-10dB above the noise level shows up with a clear color difference, however
2. If I set WF min just right, I can hide the signal in the noise inside the cyan range. I need to make cyan a little narrower. I should look into that tomorrow.

[G8JNJ]

Hi Rik,

I loaded the last couple of patches and that's a lot better :-)

However I'm concerned because of the comments made by icm7216, which I may have misunderstood.

Can you confirm that the changes only affect the colour range of the waterfall and that adding the sqrt() function does not mess up the dB scale on the spectrum display, or S meter ?

The great thing about SDR's in general is that they have a calibrated dB scale which is very useful for all sorts of signal comparison purposes.

Any change to the dB display calibration is NOT desirable and outweighs any advantage that may be gained by using the sqrt() function.

If this is the case would it be possible just to modify just the colour map values to incorporate sqrt() modified values without having to use sqrt() funtion ?

If you do have to use the sqrt() function then either 1. only apply it to the waterfall where there is no dB scale or 2. modify the dB scale to match the sqrt() function.

Regards,

Martin - G8JNJ

[G8JNJ]

Ooops,

Just noticed that the file I had modified to include the latest colour map didn't have the sqrt() function line.

I've now added it and the 'grainy' waterfall has reappeared. Not so good after all :-(

The remaped colours look better than the originals, but the problem would seem to be that the sqrt() function is a bit too severe WRT the expansion of low values.

Would it be possible to add something like a variable 'gamma correction' value ?

https://en.wikipedia.org/wiki/Gamma_correction

Regards,

Martin - G8JNJ

Regards,

Martin - G8JNJ

[G8JNJ]

Here are some examples.

First the modified colour map with the sqrt() function

Next the modified colour map without the sqrt() function

And finally the same but with different Max and Min waterfall values

So I can now see that the sqrt() function messes up the dB scale, which is a pity.

I don't think there is anything I can spot with the sqrt() function that can't be seen without it. In fact some of the weaker signals hidden among the stronger ones sometimes actually seem more obvious without it. Or is it just me :-)

For the time being I have decided to continue to trail the new colour map but without using the sqrt() function.

Regards,

Martin - G8JNJ

[rikvanriel]

I totally missed that part of icm7216's comment. Ugh.

Obviously that problem needs to be fixed. The scale on the Spectrum display needs to remain accurate.

[jks-prv]

You would think that simply altering the waterfall color map could in no way alter the dB values presented on the spectrum display (that's what I thought anyway). But it turns out it does as a consequence of the optimized code that creates the color "banding" per 10 dB on the spectrum. I'm looking at that code now trying to remind myself how it works (routine mk_dB_bands() that constructs the spectrum color map called "spectrum_data").

[jks-prv]

Sorry, never mind -- the spectrum values are effected because the sqrt() is in waterfall_color_index() and not just because of the color map construction changes in mkcolormap(). But I think this should be easy to fix. The spectrum code doesn't have to use the same waterfall_color_index() routine that is now non-linear.

[icm7216]

I think that sqrt() function is very useful for weak signal sensing. But as above they already mentioned, I am concerned about that the spectrum scale and signal level on intermediate area does not matched.

So I tried separating the color of waterfall and spectrum, but not get good result yet.

[jks-prv]

I just checked-in a change so you can say kiwi:8073/?cmap=1 to select the new colormap scheme for testing. Leave it out (or set to zero) for the original scheme. Should make it easier to do A/B comparisons. You could make other values work to test multiple schemes as well.

[icm7216]

tested on latest commit.
"cmap" option allows to compare both different kiwi, it's helpful!
Thanks.

another new color-map has lifting the spectrum.

[G8JNJ]

Hi,

Do I also need to amend the code to load in the new colour maps info, as the new cmap option doesn't seem to make any difference to me ?

Regards,

Martin - G8JNJ

[rikvanriel]

Looking at a few graphs and looking at where the noise floor and weak signals typically are, I have a few observations:

1. the noise floor is typically between 10% and 30% from the bottom of the spectrum (WF min)
2. the weakest signals we can hear are typically about 5dB stronger than the noise floor, these need to be visible
3. very strong signal peaks is anything from about 50% up in the spectrum, we don't really care about distinguishing between something 30dB above the noise floor and something 60dB above noise

I suspect we have two ways forward:

1. keep the waterfall_color_index linear, but simply adjust mkcolormap to generate a colormap with narrower bands for blue, teal, and green, bringing yellow down to maybe 45% down from the bottom, so colors change faster lower in the spectrum
2. use the square root to map signal levels to the color map, at that point we automatically get what we want in blue and teal, but we may want to narrow green a little bit

Doing both simultaneously is too much, and results in signals that are only about 1/3 of the way up the spectrum being rendered as "signal peak" colors, as well as the noise floor being an angry yellow among parts of the spectrum.

[rikvanriel]

I have put two patches in my tree to help experiment with this some more.

The first patch makes the power levels in the spectrum display linear again.

The second patch makes it possible to experiment with the sqrt color mapping, and the new color map independently from each other:

default           http://kiwisdr.yourdomain/
test colormap     http://kiwisdr.yourdomain/cmap=1
sqrt signal level http://kiwisdr.yourdomain/sqrt=1
both              http://kiwisdr.yourdomain/cmap=1&sqrt=1

To get these patches into your tree, just do this from /root/Beagle_SDR_GPS on your kiwisdr:

# git remote add rikvanriel https://github.com/rikvanriel/Beagle_SDR_GPS
# git pull rikvanriel HEAD
# make install
# service kiwid restart

[G8JNJ]

Hi Rik,

Sorry my Linux skills are limited.

I get this error message.

error: Your local changes to the following files would be overwritten by merge:
web/openwebrx/openwebrx.js
Please, commit your changes or stash them before you can merge.
Aborting

So I have to commit or stash to Github before proceeding further, but I'm not sure if this should be to a repository of my own or if it will go to yours.

Regards,

Martin - G8JNJ

[jks-prv]

If you don't care about saving your uncommitted changes, then after the "cdp" do a "m git". This is defined in the Makefile and cleans your clone of local mods before doing a pull (this is what the update process uses):

invoked by update process -- alter with care!

git:
# remove local changes from development activities before the pull
git clean -fd
git checkout .
git pull -v

p.s. if there is a better way of doing this please let me know. I'm new to git and it just makes my head hurt..

[jks-prv]

Forgot to add: this will work because I just merged Rik's latest changes into the master.

[G8JNJ]

Hi,

Great thanks for all the help.

I think Riks commands should have been.

default http://kiwisdr.yourdomain/
test colormap http://kiwisdr.yourdomain/?cmap=1
sqrt signal level http://kiwisdr.yourdomain/?sqrt=1
both http://kiwisdr.yourdomain/?cmap=1&sqrt=1

I think the new colour map on it's own makes a big difference and should work well with the majority of the SDR's including those that have a much lower maximum signal level <-50dB.

I think the square root function is too severe, even just by itself.

Note - I may be wrong about this, but I think part of the problem may be that the display is already logarithmic as it's in dB. So adding a square root function on top of this is likely to be excessive.

Black shows the log curve (dB)
Red shows what I think you wanted to achieve
Blue shows what I think you have achieved

Are there any other simple functions that could be tried instead ?

Regards,

Martin - G8JNJ

[rikvanriel]

I suspect that wanted adjustments could be achieved by simply changing the color map. How is the new color map working for people? Are there any colors that are over or under represented on your screen? Does it need more blue or yellow? Less teal or green?

Just let me know, and I can adjust the color map for you to try out.

[G8JNJ]

Hi Rik,

I think the latest map is pretty good and certainly an improvement on the original. The sliders still need some adjustment from band to band when zoomed in, but it's a lot better for picking out the weaker signals.

The utilised dynamic range on my SDR is a bit larger than most, so it's perhaps not reperesentitive of the majority of SDR's, but I'd think that if it works for me it should be even better for the rest.

Have these tests have been communicated to other KiWi admins more widely than just via Github ?

Regards,

Martin - G8JNJ

[jks-prv]

I spent some time with the new color map. I like it. Seems to make a good visual S/N improvement on the waterfall at VLF on the Kiwi here. I'm going to enable it as the default for the v1.18 update about to go out.

[rikvanriel]

When the sliders need adjustment, what colors are on your screen? In other words, which color band is so wide that the signal and noise have the same color?

Or do you always see some contrast, but are just unhappy with the colors when you adjust the sliders?

The latter I cannot address, but the former can be fixed by adjusting the color map, by making some colors narrower, and others wider.

[G8JNJ]

Hi Rik,

The colours are fine.

My problem is that my dynamic range is wider than most KiWi's and also the noise floor is higher on the LF bands in comparison to the HF bands.

So when I'm trying to see very weak signals I like to set the WF min so that the background is very dark blue. The threshold for this is very narrow and the noise level varies as I go up in frequency. Even if you further modified the colour map I think I'd still have to adjust it to acheive the same results.

Incidentally I tried the sqrt function on the weak NDB signals this evening and for me it makes no improvement to the visibility of the Morse call sign ID's. So I think you have got the colour map correct.

Regards,

Martin - G8JNJ

[jks-prv]

I'm thinking out loud here a little bit. An interesting color map to try would be one that has not a progression through the rainbow spectrum necessarily, but contains more "target" color points with smooth transitions between them (not quite sure how to say this). Something where each pair of transition points has a high contrast change. Like yellow to purple, followed by purple to green. This would give more color differentiation within, say, 10 dB gain bands while hopefully not making the noise floor too "grainy". I have seen color maps like this someplace.

Like Martin I find myself adjusting WF min/max for the integrate extension (renamed from "audio FFT" in the latest release) so the small difference between the noise floor and the integrated signal is stretched over a portion of the color map that has the best contrast. For me this has always been in the light blue region although sometime right around the yellow/red transition works too.

Here are some screenshots from the Baudline signal processing program that have some interesting ideas. The first is their "color picker" for constructing arbitrary color maps, although it is rainbow-based not the multi-point thing I was suggesting. The second shows how they have a list of pre-defined colormaps you can pick from when mapping input to drawing functions.

[rikvanriel]

That is a very good point. The transitions from teal to green, and from green to yellow, are rather hard to distinguish indeed. The various shades of blue are fairly difficult, as well.

These color transitions all share one property: only one of the Red, Green, or Blue channels is changed at a time in our current color map.

I am traveling this week, but I should experiment with other color schemes when I get back. Color transitions where multiple channels of R/G/B get changed at once may well work a lot better than the basic color map we use now.

[icm7216]

Wow good job, Thanks, Rik!

New color map is good. I can see clearly the weak signal.
In SQRT function, weak signal seems to be buried among yellow signal. I felt that disturbs the weak signal viewing. The SQRT function seems to be a little strong amplification.

[icm7216]

I wrote an improved version of the SQRT function. (see: SQRT function applied to restricted range #53) that New function was merged. Please try it.

ssh login into the KiwiSDR in advance.

cdp
git pull origin
mst
mi
msa

cdp: change to build directory
mst: stop Kiwi server
mi: make install
msa: start Kiwi server

bring up a web browser

• kiwi:8073/ => linear (default)
• kiwi:8073/?sqrt=1 => Rik's SQRT function
• kiwi:8073/?sqrt=2 => fn1, apply to between 21% and 50% spectrum range
• kiwi:8073/?sqrt=3 => fn2, apply to between 31% and 60% spectrum range
• kiwi:8073/?sqrt=4 => fn3, apply to between 41% and 70% spectrum range

kiwi as (e.g. 192.168.1.123 or kiwisdr.yourdomain)

notice

In use of new function, that helps in observation of a weak signal, but does not too much help it on the strong signal.

[G8JNJ]

Hi,

fn1, apply to between 21% and 50% spectrum range works the best for me, but this may vary for other KiWis as their signal levels are generally a lot lower than on mine.

I wonder if 0.1<x<0.3 is worth trying ?

Regards,

Martin

[rikvanriel]

I think this is something we could auto-detect.

Due to the way the waterfall code works, there is some noise in the noise floor detection, but it should be possible to assume that the noise floor is at the level below which only 5% of the detected signal level is. In other words, the 5th percentile of what is detected in the spectrum being displayed.

We also know what the WF min and WF max settings are, giving us a scaled percentage.

Now we know roughly where along the scale the noise floor is, and we can pick (or generate) a corresponding sqrt function that enhances the signals near the noise floor, putting the estimated noise floor near the lower quarter of the enhanced range, with 3/4 of the enhanced range available for the weak signals near the noise floor.

That gives us a fairly wide range in which we can be off in our estimate, while still having a wide range available to enhance weak signals.

Does that make sense?

[icm7216]

Hi Martin,

0.1<x<0.4, This can be tried right now.

cdp

Edit openwebrx.js. Add few lines under the case 4: of a function do_waterfall_index().

mst
mi
msa

openwebrx.js

function do_waterfall_index(db_value, sqrt)
{

...

    switch (+sqrt) {

    case 0:
        break;
    case 1:
        value_percent = Math.sqrt(value_percent);
        break;
    case 2:
        if (value_percent > 0.21 && value_percent < 0.5)
            value_percent = 0.2 + (4 + Math.log10(value_percent - 0.2)) * 0.09;
        break;
    case 3:
        if (value_percent > 0.31 && value_percent < 0.6)
            value_percent = 0.3 + (5 + Math.log10(value_percent - 0.3)) * 0.07;
        break;
    case 4:
        if (value_percent > 0.41 && value_percent < 0.7)
            value_percent = 0.4 + (6 + Math.log10(value_percent - 0.4)) * 0.055;
        break;
+   case 5:
+       if (value_percent > 0.11 && value_percent < 0.4)
+           value_percent = 0.1 + (3 + Math.log10(value_percent - 0.1)) * 0.125;
+       break;
}

...

then bring up a web browser

• kiwi:8073/?sqrt=5 => apply to between 11% and 40% spectrum range

Thanks,

[icm7216]

Hi Rik,

but it should be possible to assume that the noise floor is at the level below which only 5% of the detected signal level is. In other words, the 5th percentile of what is detected in the spectrum being displayed.

Unfortunately I can't understand that the assumed noise floor level 5% means. I think that 5% which cannot reasonably. Just maybe Is that only make a provisional assumption that 5%?

Because my native language is not English, could you please explain clearly in simple English?

my auto-detect idea is here.
#53 (comment)

Thanks,

[G8JNJ]

Hi icm7216 (sorry I don't know your name),

Thanks for the code.

I tried the new fn 5 patch.

It's too severe and produces a very grainy display, very similar to the original Sqrt function.

fn 1 is still the best for me 👍

Regards,

Martin