splitting out cairo backend

[bjarthur]

@tkelman has suggested that we move the cairo backend code to a separate package. could someone please walk me through the problem?

first, why is Cairo not in REQUIRE? i found the commit which removed it, but no explanation.

second, how does separating the backend fix anything? would we no longer have to Pkg.add("Cairo/Fontconfig") manually? would Gadfly and Compose go all green in the package directory?

i'm willing to do some legwork here, but don't understand well enough.

[lobingera]

I have to say i'm not fully aware of the problem described in the comment on the backports PR, to me it looks like the problem was Immerse not testable because this has Compose/Gadfly as requirements etc.

The model how required packages are imported/used/run changed over time. Something, which is still unsolved (imho) is the use of optional packages or optional dependencies (recently: https://discourse.julialang.org/t/optional-dependencies-requires-jl/3294). For Compose the original author thought of Cairo/Fontconfig as optional dependencies.

So Compose has a device independent part and (roughly) two backends: Cairo based and local code (for SVG).
So i think, @tkelman suggestion is: to split into: ComposeDVI and ComposeCairo. Gadfly would then require ComposeDVI and only import ComposeCairo if requested by selection of output device.

[lobingera]

btw: testing of Compose in pkg fails, because the test does a binary comparison of output files and that's a major problem, i.e. #222

[lobingera]

testing of Gadfly fails out of the 'required dependency' reason

[tkelman]

The problem is that the decision of which backends to enable gets made at precompilation time, but at use time the available options can differ.

The Immerse issue is due to Compose's badly implemented conditional dependency on Cairo breaking down with precompilation. If you use a clean JULIA_PKGDIR, install and precompile Compose alone (without Cairo), then install and try to import Immerse, it fails.

Immerse works fine if you have Cairo installed at the time you precompile Compose. It breaks if you precompile Compose at a time when Cairo is not installed. Installing Cairo does not invalidate the previous precompile cache, so Compose is actually not precompile safe.

[bjarthur]

am i missing any of the options we have to resolve this issue in the list below?

1. add Cairo to REQUIRE. downside here is a dependency on a non-native library which might not be able to be installed on all systems.

2. delete Cairo entirely because SVG is soo much better, and SVG can always be converted to PDF, PNG, etc. by some third party utility

3. split cairo_backend.jl out into a separate package, which the user must then explicitly include should they want PDF, PNG, etc. output

4. don't precompile Compose

5. wait until julia supports optional/conditional dependencies to do anything. we do have limited resources here, which might better be served fixing other bugs / adding new features, instead of having to fix this once now, and a second time later.

thanks for the link to the particularly informative discourse discussion.

[tkelman]

That's a pretty good list of possibilities, yes.

Revisiting #226 could be one other possible option but I don't think Requires would actually fix the precompilation problem on 0.5 or 0.6 so it might not be any better than option 5.

[bjarthur]

#139 is related

[tkelman]

yes, it's been a long standing mistake - if people have to manually manage their cache to fix inconsistencies, the package isn't precompile safe at all

[tlnagy]

We now do conditional loading of the Cairo code with #282. There are still some issues but those are tracked by #299