Relative paths computed incorrectly in attribute expansions

[wolandscat]

Observed vs. expected behavior

Background: we have a collection of standards, each is an Asciidoctor multi-file book, and the books for various 'components' are in multiple GitHub repos, one for each component. The files in the books use many variables that come from files within .adoc files in one special component repo. All this works fine with Asciidoctor 2.x (for some years), Ubuntu 22.

In each component Git repo, the documents are in directories below a /doc directory.

I have IntelliJ IDEA 2021.2.2 (Ultimate), with the AsciiDoc plug-in 0.36.19. Installation of plug-in appears to be fine, diagram module downloaded, basic functionality as documented by this project.

If I open the /docs directory in IntelliJ, and then navigate to a master.adoc document in one of the /docs/xxx subdirs, the plugin makes a pretty good attempt at processing all the includes (of chapters, and below chapters, of smaller files). It's correctly processing at least some variables.

Variables that would normally be passed to asciidoctor on the command line are in a new file /docs/.asciidoctorconfig, created according to plug-in documentation. This is being read, because I can see some of the variable substitutions are working.

The main problem is that image paths containing Asciidoc attributes are not being processed correctly. (Image rendering does appear to be working because some image paths are external URLs, and they resolve to SVGs that do display in the previewer).

Example: the following line is how we commonly include a UML diagram (an SVG file).

image::{uml_diagrams_uri}/AM-aom2.constraint_model.svg[id=constraint_model, align="center"]

The attribute uml_diagrams_uri is defined to have the value UML/diagrams in a file that is included by master.adoc.

If I send the output to FireFox instead of the internal browser, and then inspect the img path, I see this:
<img src="image?file=%2Fhome%2Fthomas%2FopenEHR-specifications%2Fspecifications-AM%2Fdocs%2FAOM2%2FUML%2Fdiagrams%2FAM-aom2.archetype.svg&mac=lJGABj9KNIyoSiSrBUcY5W+JrtQAr7OILhAYaCkTm/A=&hash=none">

However, if I use asciidoctor on the command line, (via our scripts), the same image has the following HTML <img src="UML/diagrams/AM-aom2.archetype.svg" alt="AM aom2.archetype">

The Asciidoc plugin appears to be prepending a root path to the relative path that is defined in the Asciidoc source, as well as doing other things I don't understand.

The other main problem is that if I click on any chapter file (i.e. a file included by master.adoc), only the text is rendered - anything depending on vars coming from master.adoc doesn't work. This could be faked by more .asciidcotorconfig files I guess but that's far from optimal, and likely to be too much trouble to be worthwhile. A better solution would be some special variable telling the plug-in that files with name master.adoc (or whatever name) are book files, and to look for them in the directory of any other .adoc file in order to obtain all the attributes and inclusions.

Although the above directory / file structures may sound a bit complex, they are common in multi-document publishing, as is the use of multi-level inclusion of files containing variables and so on.

I think if the plug-in can succeed at rendering these files, it can do anything. I'm sure the problem reported here is not complex to fix.

Steps to reproduce

Hopefully the above gives an idea of the problem. Since all the documents are open source, you could reproduce using the following steps.

1. create a directory openEHR (NB - I do all this on Ubuntu; should work on Windows, but you'll have to work out the adjustments)
2. clone https://github.com/openEHR/specifications-AA_GLOBAL (the 'special' component containing some include .adoc files) under it, i.e. creating openEHR/specifications-AA_GLOBAL
3. clone https://github.com/openEHR/specifications-AM in the same way, as a sibling. This project contains some docs that can be used to demonstrate the problem
4. create the file /docs/.asciidoctorconfig with contents shown below this list
5. open IntelliJ; open specifications-AM/docs; this will get you a view on the left of docs with names like ADL1.4, AOM1.4, AOM2 etc
6. navigate to AOM2/master.adoc and double click. It will render in the internal browser; navigate to section 3.1 in the browser view - you will see a broken diagram link below the first para. (Figure 6).
7. In the editing pane, send the preview output instead to FireFox (or other external browser) and navigate to the same diag, then inspect to see the (wrong) generated img HTML.

Contents of /docs/.asciidoctorconfig; replace <root> with /absolute/path/to/openEHR dir created above.

:component: AM
:pub_home: <root>
:work_root: <root>
:ref_dir: <root>/specifications-AA_GLOBAL
:base_dir: <root>/specifications-BASE
:stylesdir: https://specifications.openehr.org/styles

:allow-uri-read:
//:bibtex-file: {ref_dir}/docs/references/references.bib

Environment

Ubuntu 22 (20 will be fine)
Plugin Version: 0.36.19
IntelliJ Details: IntelliJ IDEA 2021.2.2 (Ultimate)

[ahus1]

Thanks for asking this question on the issue tracker.

In the setup you describe above, I assume you open the folder openEHR as the project in IntelliJ. If not, I recommend you should, as this allows you adding a .asciidoctorconfig file in the openEHR folder.

That file should contain the following contents:

// avoid errors for the include by setting a default; asciidoctorconfigdir is only set when using it as a prefix 
ifndef::asciidoctorconfigdir[:asciidoctorconfigdir: .]

:pub_home: {asciidoctorconfigdir}
:work_root: {asciidoctorconfigdir}
:ref_dir: {asciidoctorconfigdir}/specifications-AA_GLOBAL
:base_dir: {asciidoctorconfigdir}/specifications-BASE
:stylesdir: https://specifications.openehr.org/styles

// re-use attributes from other places
include::{asciidoctorconfigdir}/specifications-AA_GLOBAL/docs/boilerplate/global_vars.adoc[]

Still, this won't render any images in the preview. To make that work, the image macros need to use a relative path to point to the image. {uml_diagrams_uri}/AM-aom2.archetype.svg will resolve to UML/diagrams/AM-aom2.archetype.svg, so it is missing a ../ to find the image. To fix that, you can used the imagesdir attribute and set it in an .asciidoctorconfig file in the same folder as the master.adoc file:

:imagesdir: ../

Unfortunately this will only fix images in that folder; I see that there are a lot of images in different folders, also the AA_GLOBAL structure. As only one imagesdir can be defined, this will never work for all folders. As the imagesdir is a relative path, you'd need to define it in every folder where there is AsciiDoc content that references images.

I'm working on a change that in the next version of the AsciiDoc plugin, you can use a file-URL, so a setting like the following would work:

:imagesdir: file:///{asciidoctorconfigdir}/docs

In the case of this document, you could even set uml_diagrams_uri to such a value, and not set imagesdir -- this would allow you to have different file types in different folders.

Please let me know if you think this goes into the right direction although it might not yet be the full solution in your scenario.

[wolandscat]

Thanks for the response, and thanks for providing the plug-in - I can see it is pretty solid, and these directory things are tricky - but I'm sure you can iron out the issues, and it will be very nice indeed.

In my first report I missed something - the /docs dir is in openEHR/specifications-AM, and that is what I open in IntelliJ, and it's also where the .asciidoctorconfig file is. I'll now do it the way you propose.

Another thing I neglected to mention is that those relative image directories like UML/diagrams are like that because the final HTML doc is not written to the source files directory, but to its parent, i.e. all the .html files end up just in /docs, which is where you want them. Those relative paths then all resolve correctly when the docs are served locally within a browser, or from a server e.g. Apache or similar. I.e., the structure is:

/docs
    doc_A.html
    doc_B.html
    /doc_A
        master.adoc
        master00-xxx.adoc  # chapter files
        ...
        master0N-yyyy.adoc
    /doc_B
        etc
    /UML
        /diagrams
        /classes

You can see that the file doc_A.html can have an <img src="UML/dagrams/xxx.svg"> and it will work.

All of the above comments apply equally to any paths - not just img paths.

Generally speaking, a publishing environment like this will want to be able to a) put the generate HTML docs in a place different from the source files directories (normally, in some parent location, to make shorter paths in URLs); and b) to be able to use relative paths in HTML img and href paths, for web server efficiency.

The challenge for a tool like the plug-in is that it is trying to render the equivalent HTML into the same directory as the source files, and of course those local paths will not be correct.

So I think the solution is something like a) generate any relative paths into the HTML as is (as Asciidoctor does) - don't try to mess with them and b) generate the HTML into a specified target directory (not '.'). In our case, it would always be '../'.

How does that sound?

[ahus1]

Hi @wolandscat - while the setup you describe in the first post is a real-world setup, a simpler example focused on showing a specific need of problem could help me to give answers that are more to the point. Your last comment with a smaller example helps me with that.

In the smaller example in your previous comment, I read that all AsciiDoc files are in a folder like doc_A and all images they reference are given relative to the parent folder where you placed the doc_A.html

In such a setup, add a file .asciidoctorconfig with the following content:

:imagesdir: ../

Then the HTML rendered for the preview will show the images defined with paths relative to the docs folder as for example image::UML/diagrams/image.svg[].

[wolandscat]

I have done what you suggest; it does make the UML diagrams (SVGs) now show when I double-click on master.adoc (this means I have to add about 40 .asciidoctorconfig files just to make this work, plus various higher-up ones). But no other images show, because they are usually in ./images or ./diagrams, not ../images etc.

But there (as you realised) images in completely other places as well, and all the paths generate out correctly for that to work as an HTML document.

I'm not sure what the right approach is to fix this for the plug-in, but one way to think about it might be: if I double-click on master.adoc, and then click on one of the external browsers as the target, that browser has to receive a .html file location (presumably it's currently in some temp location?). But if the html document were in the expected place that is assumed by the publishing system (mine, someone else's, Antora, ...) then it will render properly, including all diags. So then it may make sense to treat the built-in browser in the same way - give it the right location of the html output so all the relative paths resolve.

[ahus1]

If the settings are the same for all folders, you can move .asciidoctorconfig file upwards.

I see that different image types have different attribute prefixes in their definition, like {diagrams_uri}. You can redefine those attributes in the .asciidoctorconfig so that it finds diagrams in the diagrams folder, while all other images in are another folder.

Other projects have used symlinks for common resources, still that doesn't fit all custom projects.

I'm afraid this is all the plugin can offer for a custom project setup. I hope one of the tools or a combinations leads to a workable setup for you.