PROJ, proj-data, and accurate reprojections

[brynpickering]

I have been wondering for a while why my local build fails a bunch of tests every time and it turns out that there is a difference in the contents of the PROJ library on conda-forge and on e.g. Homebrew.

The conda-forge version sets an environment variable that leads to transformation gridfiles being pulled from the internet where possible; the homebrew version doesn't set this but downloads the gridfiles and stores them locally on installation (see here). To get these same data files, the conda-forge proj-data library also needs to be installed. It's a big boy: ~700MB.

The conda-forge PROJ has the advantage of a lower footprint on disk, but the attempt to query online gridfiles has an associated bug which I've come across (and which led me down this rabbit hole): pyproj4/pyproj#705

Oddly enough, genet tests only pass with conda packages installed if I do not install proj-data and force PROJ to use its internal (quite simple) transformation grids (pyproj.network.set_network_enabled(False) in genet.__init__.py). This suggests that we are not accessing this large transformation library by default. If our tests are running on the simplified transformations, there are possible accuracy improvements to be had...

[brynpickering]

In addition, I've checked the Docker image builds on AWS (do a search for libproj / proj-data) and the use of apt-get is leading to the old transformations being installed (PROJ v7.x) which we found led to different results than the more current versions of PROJ (>= v8.x). We discussed that issue in #167.

On the GitHub CI it is v8.x being installed.

Setting minimum versions with apt-get seems to be a pain, so this is another reason to move to a fully conda-fied install/build pipeline where we can be more explicit with version setting of PROJ and we decide if we want to include proj-data in an install.