Ability to host your own WrapDB

[LunarLambda]

I maintain a large list of meson wraps for Game Boy Advance homebrew projects.

Currently, making these available to users requires they download the meson-gba skeleton project, which includes all files for all libraries and tools unconditionally. This is both messy for the end user, and messy for me to maintain and test (currently I have a dev branch I need to keep somewhat painstakingly rebasing against master, since my development setup and the end user setup are different). It also means to get an updated version of a wrap, the end user needs to either manually download the files, or do a git stash; git pull; git stash apply dance)

Meson provides meson wrap for downloading and adding wraps from a hosted database. However, there's no mechanism to use a database other than the meson-provided WrapDB. Besides being a potential single point of failure (no ability to use mirror servers), this means I am left with two options, either reimplementing a form of WrapDB for my project, or continue doing it this way. The highly domain-specific nature of these projects means they're unlikely candidates for the official WrapDB, and even without that, the overhead of making semi-frequent pull requests to WrapDB for on the order of two dozen projects is simply not feasible for me.

It would be great if meson could allow specifying a WrapDB server, or a similar functionality, such as being able to download a wrap from a WrapDB-like git repository, for example.

[LunarLambda]

#6497 (comment) Apparently the service backing WrapDB is already open source and could (possibly?) be used for hosting, so it's just a matter of allowing to specify a source for meson wrap to use.

[xclaesse]

That's the old server implementation we are not using anymore. WrapDB v2 is much simpler, all it needs is a static URL to a releases.json file.

[xclaesse]

3rd party WrapDB has always been something we wanted to allow. It's not currently possible, but it's shouldn't be too hard to get there.

There is only a handful places in Meson where the wrapdb URL is hardcoded:
https://github.com/mesonbuild/meson/blob/master/mesonbuild/wrap/wrap.py#L111

Basically what would be needed is a command line argument to "meson wrap" to specify a custom URL where to find a releases.json file.

[LunarLambda]

gotcha.

Also, I think meson wrap downloads some .json file to the subprojects directory, right?

Could I theoretically just handwrite/generate one and ship that in the meantime? I didn't have time to investigate it, I just randomly spotted it while reading the docs. If it's good enough to let me hoist the packagefiles elsewhere, that's fine enough

[xclaesse]

meson wrap update-db writes releases.json into subprojects/wrapdb.json. That's a way for not having to do meson wrap install each wrap file individually. We should add into that json file the URL of the wrapdb instance you're using. But you'll still need to generate release tags that contains the patch overlay tarball. That's what the script I linked above does. One could of course write a server that would generate those tarballs on the flight I guess.

[LunarLambda]

So a patch for this would be twofold, one to add the CLI argument and one to add the URL field to releases/wrapdb.json?

[xclaesse]

I think when update-db subcommand writes subprojects/wrapdb.json, it whould add a field in that json with the URL from where it downloaded it. That way, in wrap.py get_from_wrapdb() you can fetch the .wrap file from the right URL.

[xclaesse]

BTW, all those https://wrapdb.mesonbuild.com/v2 URLs work with a simple server we have that redirect to github: https://github.com/mesonbuild/wrapdb/blob/master/nginx/default.