Speed up data processing for the data layer

[deevroman]

This PR doesn't fix the issue openstreetmap/openstreetmap-website#5466

It fixes a problem that will show up after it is fixed. But you can notice it now.

1. Load a large area of ​​the map and wait for the /map method to finish loading
2. You will have to wait a few seconds for the data to be processed before you see the warning.

On flame graph it now looks like this:

https://share.firefox.dev/4a3ZXM9

The error lies in the quadratic complexity of the data processing algorithm.

Suppose you have loaded 3000 nodes and they are all in ways. (it’s easy to imagine buildings in a city or roads) Now the algorithm for each loaded node makes a traversal through all nodes of all ways and members in the relations. We get 3000*3000 ~ 9,000,000 operations!

The situation is also aggravated by the fact that the array of relation members keeps members that are not nodes and you still traversal them for each loaded point!

Total:

1. Replace indexof() with a hashmaps of loaded nodes.
2. Remove null and undefined from the members of relations.

p.s. I'm not sure if you currently support new browsers, so I wrote code that does not use the new language features

[tomhughes]

Shouldn't this check that the member is actually a node? I realise the old code wasn't but that seems like it was a pre-existing bug?

[deevroman]

In the old and new code, this check is not necessary, since all members are not nodes are deleted https://github.com/deevroman/leaflet-osm/blob/d72b63467a891895877f8b121f7a9822ca7dcdf4/leaflet-osm.js#L309-L312

[tomhughes]

Thanks. In that case I guess this is good.

[mmd-osm]

It seems this change is causing some regressions: https://community.openstreetmap.org/t/transport-routes-are-displayed-without-stops-on-web-front-end/123949

For me, I also don't see bus stop nodes anymore when displaying bus route relations. I tested with commit 231177aa29c8c09135d9921956341a5f477cbb7b, where it was still working ok.

[AntonKhorev]

interestingNode is redefined here:

https://github.com/openstreetmap/openstreetmap-website/blob/f946797520d5420804a730657dab99c504cae693/app/assets/javascripts/leaflet.map.js#L315

[tomhughes]

Ugh. That should never have been allowed... It happened ages ago in openstreetmap/openstreetmap-website@a125ee8 but it should have been by adding a proper API here if necessary, not by inserting itself by the back door like that.

[AntonKhorev]

It shouldn't be necessary to redefine interestingNode. All it currently does is it makes way nodes uninteresting. Let's say there's a road and some of its nodes are traffic lights or crossings. In the leaflet-osm version those are interesting, but osm-website makes them uninteresting when viewing just the way. There could be an option for L.OSM.DataLayer that does this instead. Currently it can't make this decision because it just shows whatever is in xml it's fed, it doesn't know if the client wanted to view a particular osm element.

[tomhughes]

I've pushed openstreetmap/openstreetmap-website@70012ab as a quick fix but as you say we should find a way to do it better.