Add an Span (Inline) Comment Token?

[rossbar]

Describe the bug
Adding a line comment (starting with %) introduces a newline in the html output.

To Reproduce
Using docs at hand:

1. Open docs/examples/wealth_dynamics_md.md
2. Add a comment somewhere in the center of a paragraph, e.g. near line 117:

A value of 0 indicates perfect equality (corresponding the case where           
%TODO: Add more here
the Lorenz curve matches the 45 degree line) and a value of 1 indicates         
complete inequality (all wealth held by the richest household).

3. Build: cd docs && make html
4. Open the page in the browser and observe the corresponding section.

Expected behavior
Line comments would have no affect at all on the output text.

Screenshots
N/A

Environment (please complete the following information):

• Python Version [e.g. 3.7.1] - 3.8.2
• Package Version [e.g. 0.1.0] - MyST-Parser 0.5.1

Additional context
N/A

[chrisjsewell]

So, this is the actual docutils AST rendered:

line 1
% a comment
line 2

<document source="notset">
    <paragraph>
        line 1
    <comment xml:space="preserve">
        a comment
    <paragraph>
        line 2

This is the 'correct' behaviour, given the fact that (as it stands) a LineComment is defined as a block level entity,
such that when one is found the current block element is terminated (in this case a paragraph).

What you are after here is a span level (a.k.a inline) comment entity, whereby the comment is parsed as:

<document source="notset">
    <paragraph>
        line 1
        <comment xml:space="preserve">
            a comment
        line 2

This essentially would require an additional syntax element to be implemented, which would be fairly tricky, so I can't promise I could achieve this any time soon.

Perhaps at a minimum the documentation should make this clear.

[rossbar]

Got it, thanks for clarification. It's not fair to blame this on the documentation necessarily - I had barreled ahead without realizing the distinction between the line comment and inline comments 😑 . That said, I will take a closer look to see if an extra example in the docs wouldn't hurt.

[chrisjsewell]

No problem, I've added the documentation now in #117. Take a look at https://272-240151150-gh.circle-artifacts.com/0/html/using/syntax.html#comments