How to represent Declarator Docs in plain RakuDoc

[patrickbkr]

Prelude:

Toolchain wise we've long had the issue that tools to display documentation (GitHub, GitLab, raku.land, ...) have an issue with running the Rakudo parser on source code to reach for the embedded docs as that code can contain BEGIN blocks, which is a big safety concern.

A possible way forward is to split the task of rendering RakuDoc (and Declarator Docs in particular) into two phases:

1. A tool, the Extractor extracts and converts all docs in a Raku source file and outputs a pure .rakudoc file. This is an unsafe procedure.
2. A separate tool, a plain RakuDoc parser, then parses that pure .rakudoc file. It barfs on anything that is not plain RakuDoc. This is a safe process.

Platforms that wish to render RakuDoc (e.g. GitHub) can safely use the RakuDoc parser. Platforms that want to render the full documentation (e.g. raku.land) can run the Extractor in a sandbox.

All of the above is not part of the issue this ticket wants to address.

---

The question I want to discuss is: What should the output of the above mentioned Extractor that can convert a .rakumod file into a .rakudoc file look like?

The issue is mostly orthogonal to #438 in that it is not concerned with how Declarator Docs should look in code, but how they should be represented in RakuDoc.

[patrickbkr]

Just to get the discussion started, here is the first thing I was able to come up with.
Given this piece of code in a file called foo/bar/frobnicator.rakumod:

#| The class C<Foo::Bar::Frobnicator> is the one stop shop for all your frobnication needs.
class Foo::Bar::Frobnicator is Affector[I] does Helping {
    #| Perform a frobnication.
    method frobnicate(
        Str() $thingy = "something", #= The thing you want to frobnicate.
        :$quiet                #= Do it quietly.
    ) { ... }
}

#| User expertise levels.
enum Level
    ONE, #= Beginner.
    TWO, #= Expert.
;

#| Default config values.
my %config =
    size docs "How big should the buffer be?" => 5,
;

The output in a file foo/bar/frobnicator.rakudoc might be:

=begin dock :type("class")
=           :name("Foo::Bar::Frobnicator")
=           :code("class Foo::Bar::Frobnicator is Affector[I] does Helping")
=           :file("foo/bar/frobnicator.rakumod")
The class C<Foo::Bar::Frobnicator> is the one stop shop for all your frobnication needs.
=end dock

=begin dock :type("method")
=           :name("frobnicate")
=           :code("method frobnicate(Str() $thingy = \"something\", :$quiet)")
=           :file("foo/bar/frobnicator.rakumod")
=           :context(class => "Foo::Bar::Frobnicator")
Perform a frobnication.
=end dock

=begin dock :type("parameter")
=           :name("thingy")
=           :code("Str() $thingy = \"something\"")
=           :file("foo/bar/frobnicator.rakumod")
=           :context(class => "Foo::Bar::Frobnicator", method => "frobnicate")
The thing you want to frobnicate.
=end dock

=begin dock :type("parameter")
=           :name("quiet")
=           :code(":$quiet")
=           :file("foo/bar/frobnicator.rakumod")
=           :context(class => "Foo::Bar::Frobnicator", method => "frobnicate")
Do it quietly.
=end dock

=begin dock :type("enum")
=           :name("Level")
=           :code("enum Level")
=           :file("foo/bar/frobnicator.rakumod")
User expertise levels.
=end dock
=begin dock :type("literal")
=           :name("ONE")
=           :code("ONE")
=           :file("foo/bar/frobnicator.rakumod")
=           :context(enum => "Level")
Beginner.
=end dock
=begin dock :type("literal")
=           :name("TWO")
=           :code("TWO")
=           :file("foo/bar/frobnicator.rakumod")
=           :context(enum => "Level")
Expert.
=end dock

=begin dock :type("variable")
=           :name("config")
=           :code("my %config")
=           :file("foo/bar/frobnicator.rakumod")
Default config values.
=end dock
=begin dock :type("key")
=           :name("size")
=           :code("size")
=           :file("foo/bar/frobnicator.rakumod")
=           :context(variable => "config")
How big should the buffer be?
=end dock

Immediate questions:

• Which syntactic elements do we allow Declarator Docs on? (Declarator Docs should be limited in scope #438 might provide the answer for that.)
• Which components of some piece of code do we want to expose as separate metadata keys and which do we not want to expose? E.g. thingy could have :positional(0), :default("something"), :sig-type("Str()") and so on. How much is too much?
• Would the above be enough to provide a decent rendering of the above Declarator Docs?
• I have the feeling that :context and :file can not be as simple as I make them be in the above example. Are there conditions in which the naive approach above fails?

@finanalyst I believe you currently have the deepest insight into implementing renderers. Thus the ping.

[tbrowder]

I forgot but just found that back in 2020 we did get the ability to keep the leading declarator block in its original form with use of an environment variable RAKUDO_POD_DECL_BLOCK_USER_FORMAT. That implementation was the quickest and easiest way to affect Raku parsing.

When that is true, and you run 'raku --doc ...', you will get text from the leading declarator blocks as input by the author of the code.

I used that for documenting my code. I hope that is retained and improved for RakuAST.

The test for it is in roast file 'S26-documentation/block-leading-user-format.t'. I

[lizmat]

It is retained if you use $=pod.

[lizmat]

What I implemented for --rakudoc now:

#| before subset
subset Positive of Int where * > 0; #= after subset

#| before enum
enum Frobnicate <Yes No>;  #= after enum

#| before regex
my regex zippo { z }  #= after regex

#| before block
{ … } #= after block

#| before pointy block
{ … } #= after pointy block

#| the main class
class A {  #= that we have

    #| a method before
    multi method a( #= method after
      #| parameter before
      Int:D $foo #= parameter after
    ) {
        #| a variable
        my $a = 42; #= with the answer
    }
}

produces:

=begin doc-subset :name<Positive>
  =leading before subset
  =trailing after subset
=end doc-subset

=begin doc-enum :name<Frobnicate>
  =leading before enum
  =trailing after enum
=end doc-enum

=begin doc-regex :name<zippo>
  =leading before regex
  =trailing after regex

=end doc-regex

=begin doc-block
  =leading before block
  =trailing after block

=end doc-block

=begin doc-block
  =leading before pointy block
  =trailing after pointy block

=end doc-block

=begin doc-class :name<A>
  =leading the main class
  =trailing that we have

  =begin doc-method :multi :name<a>
    =leading a method before
    =trailing method after

    =begin doc-parameter :name<Int:D $foo where { ... }>
      =leading parameter before
      =trailing parameter after
    =end doc-parameter

    =begin doc-variable :name<$a>
      =leading a variable
      =trailing with the answer
    =end doc-variable
  =end doc-method
=end doc-class

[lizmat]

Note that this was just a first attempt and no way intended to be cast in stone... Very much open to improvements :-)

[patrickbkr]

@lizmat It's a nice surprise you actually started working on it already. I did not see that coming. You've implemented support for quite a list already. Nice!

The first elements that come to mind I still miss are: doc-sub, doc-constant, doc-has, doc-token, doc-rule, doc-grammar

Questions:

• Is there any point in putting a Dock on a block? I can't imagine a plain block to be part of an API. It's especially strange given that blocks have no name or other identifier by which they could be referred to.
• Same can be asked for my variables. No, it makes perfect sense - e.g. for IDE support for maintainers.
• Nesting the hierarchies is nice. Are there cases where this might not be easily possible? E.g. When having a class spread multiple files using augment?
• Can we have the literal code of the respective element in there? (What I put in :code() in my attempt.)
• Semantically, do we want to give #| and #= different meanings? If not, then we can get rid of =leading and =trailing and just have the contents directly in there.

[lizmat]

I still miss: doc-sub, doc-constant, doc-has, doc-token, doc-rule, doc-grammar

Hmmm looks like decks on constants are currently broken.

Re: doc-sub these all fall out of generic Routine handling

#| before sub
sub foo() { #= after sub
}

produces:

=begin doc-sub :name<foo>
  =leading before sub
  =trailing after sub

=end doc-sub

But it looks like submethod is being rendered as "doc-method" instead of "doc-submethod"

Looks like has is being rendered as "doc-variable" instead of "doc-has". Hmmmm...

"doc-token" and "doc-rule" are already being rendered correctly, as they are a special case of "doc-regex".

"doc-grammar" and all other package types, are already being rendered.

[lizmat]

Is there any point in putting a Dock on a block?

There are spectests for it.

[lizmat]

Same can be asked for my variables. No, it makes perfect sense - e.g. for IDE support for maintainers.

And there the line between internal and external documentation blurs. So are decks for external or internal documentation?

[lizmat]

Nesting the hierarchies is nice. Are there cases where this might not be easily possible? E.g. When having a class spread multiple files using augment?

All RakuDoc is based on a RakuAST tree. However you build such a tree, is open to debate. I guess one could join multiple ASTs from different files into a single AST. Then it all depends on the scoping of such a merge.

[lizmat]

Can we have the literal code of the respective element in there? (What I put in :code() in my attempt.)

Yes, you could. But this could become quite a lot. And wasn't it the point to not have any code in safe RakuDoc?

[lizmat]

Semantically, do we want to give #| and #= different meanings? If not, then we can get rid of =leading and =trailing and just have the contents directly in there.

Sure, we could do that.

[lizmat]

rakudo/rakudo@27565cc1f7 now renders attributes as doc-attribute.

[lizmat]

rakudo/rakudo@8162f3eb3b now renders submethods as doc-submethod.

[patrickbkr]

The point is to not have any Raku code that needs to be parsed. But a string containing some Raku code (i.e. no different from a =code block in RakuDoc) is not a problem. My feeling is, that the code (especially for routines - the declarator) carries a lot of meaning in itself and would help a lot to have it available in the docs. For each Signature parameter, the ':' tells if it's a named arg, the '!' tells, if it's mandatory, the order of positional args is clear, the types tell what you're meant to pass in, ... I think that's the reason why [in many places](https://docs.raku.org/type/Proc/Async) in the official Raku docs the routine declarators are shown. Alternatively we could expose all the different bits of information separately. In the case of Signature parameters that might look like: =begin doc-parameter :name<foo> = :mandatory = :type<Int:D> = :container<Scalar> = :kind<Positional> = :position(0) = :where(...) = :aliases(...) = :default(...) parameter before parameter after =end doc-parameter It's definitely more work to provide the information separately instead of just giving `:code<Int:D $foo>`, but could possibly simplify the job of the consumers of this data. In the end most renderers will want to display something like `Int:D $foo` to the user. In my ideal world I'd love to have a cleaned up version of the code where comments are removed, whitespace is normalized and stuff like `name($!bind-var)` is cleaned up (i.e. that `!` should be removed).

…

On September 22, 2024 1:01:39 PM GMT+02:00, Elizabeth Mattijsen ***@***.***> wrote: > Can we have the literal code of the respective element in there? (What I put in :code() in my attempt.) Yes, you could. But this could become quite a lot. And wasn't it the point to **not** have any code in safe RakuDoc? -- Reply to this email directly or view it on GitHub: #439 (comment) You are receiving this because you authored the thread. Message ID: ***@***.***>

[patrickbkr]

That's a question that is difficult to answer. There is clearly a use case for both. Question is, which of the use cases do we want to support? My feeling is that explicitly excluding one of the two use cases diminishes the usability. I would say, let's just leave it unspecified for now, provide a generic tool and see how it's used and what works and what doesn't. If we find, that there is a need to separate the two uses more strongly, we can adjust later on.

…

On September 22, 2024 12:58:38 PM GMT+02:00, Elizabeth Mattijsen ***@***.***> wrote: > ~~Same can be asked for my variables.~~ No, it makes perfect sense - e.g. for IDE support for maintainers. And there the line between internal and external documentation blurs. So are decks for external or internal documentation? -- Reply to this email directly or view it on GitHub: #439 (comment) You are receiving this because you authored the thread. Message ID: ***@***.***>

[patrickbkr]

I've just realized, that we'll probably not only want to generate content for the elements that have a deck attached, but for all elements that are part of a files public API.
So given

#| A fighter
class Warrior {
  has $.name;
  has $.hp; #= health points
  has $.strength;
}

We should also generate entries for $.name and $.strength even though they don't have any deck attached.

[lizmat]

We should also generate entries for $.name and $.strength even though they don't have any deck attached.

That could only work if the class specification itself has a deck. Otherwise it'd be hard to go back through the attribute to the class and get the sequence in the doc correct.

[patrickbkr]

That could only work if the class specification itself has a deck. Otherwise it'd be hard to go back through the attribute to the class and get the sequence in the doc correct.

Could we do it differently and extract all the wanted elements (packages, classes, methods, ...) directly instead of going backwards from the decks?

Going via the decks has the additional disadvantage, that only elements that have some deck attached somewhere in their hierarchy are exported. Deckless elements would be ignored.

[finanalyst]

At the moment, it would be useful for me at least, if we could focus on how to manage the AST that is now being generated.
A simple example with both rakudoc and a pre-deck (declarator block leading the declaration) now is

=begin rakudoc
some stuff
=end rakudoc
#| bleh
my $var;

Running on a CLI, we have

$ raku -e 'q:b[=begin rakudoc\n some stuff\n=end rakudoc\n#| bleh\nmy $var;].AST.say'
RakuAST::StatementList.new(
  RakuAST::Doc::Block.new(
    type       => "rakudoc",
    paragraphs => (
      RakuAST::Doc::Block.new(
        margin     => " ",
        type       => "implicit-code",
        paragraphs => (
          "some stuff\n",
        )
      ),
    )
  ),
  RakuAST::Statement::Expression.new(
    expression => RakuAST::VarDeclaration::Simple.new(
      sigil       => "\$",
      desigilname => RakuAST::Name.from-identifier("var")
    ).declarator-docs(
      leading => (
        "bleh\n",
      )
    )
  )
)

If we filter out just the RakuDoc (the .rakudoc method on an AST), which is what a renderer will do, we get:

$ raku -e 'q:b[=begin rakudoc\n some stuff\n=end rakudoc\n#| bleh\nmy $var;].AST.rakudoc.say'
(RakuAST::Doc::Block.new(
  type       => "rakudoc",
  paragraphs => (
    RakuAST::Doc::Block.new(
      margin     => " ",
      type       => "implicit-code",
      paragraphs => (
        "some stuff\n",
      )
    ),
  )
) RakuAST::Doc::Declarator.new(
  WHEREFORE => RakuAST::VarDeclaration::Simple.new(
    sigil       => "\$",
    desigilname => RakuAST::Name.from-identifier("var")
  ).declarator-docs(
    leading => (
      "bleh\n",
    )
  ),
  leading   => (
    "bleh\n",
  )
))

I don't have much idea about how this might look.

Suppose that all declarations are gathered together into a Table, perhaps called DECTABLE, is treated as an implicitly hidden semantic block. Then any where inside a Rakudoc source we could have
=place semantic:DECTABLE :caption<Declaration blocks>, but for a default, we place it at the end.
Then we might get something like:

name	type	description
$var	Simple	bleh

As for security, here is a speculation:
In order to make this completely safe in a fairly general case including BEGIN block, I think - let me know if I'm wrong - that we could generate the AST of a source (a rakumod or rakudoc file) inside a secure container, serialise the AST.rakudoc, then render from the deserialised AST.rakudoc.

[patrickbkr]

Suppose that all declarations are gathered together into a Table, perhaps called DECTABLE, is treated as an implicitly hidden semantic block. Then any where inside a Rakudoc source we could have =place semantic:DECTABLE :caption<Declaration blocks>, but for a default, we place it at the end. Then we might get something like:
name type description
$var Simple bleh

We are dealing with two separate sets of documentation that follow a different structure.

1. The non-declarator-block RakuDoc parts follow a top to bottom ordering in a file and structured in a flat list of file names. RakuDoc provides flexible tools to introduce ordering via automatic or manually created TOCs, indexes and links.

2. The deck (declarator-block) RakuDoc parts follow the hierarchical structure of the code. The sum of all decks form a single tree of package / namespace -> class / role -> method / sub.

I think we should not try to bend decks (2.) to fit into the RakuDoc documentation (1.) structure. I think it's not possible to do this cleanly as we are working against the preexisting structure.

My current best idea is to keep the two structures separate. So when processing the documentation we'll produce two separate sets of documentation each following their own structure.

1. Resembles the output of typical documentation systems, e.g. The Sphinx docs
2. Resembles an API documentation, e.g. JavaDoc or Doxygen

That should be cleanly possible. Building on that we can allow linking to and embedding parts of 2. in 1. (Linking might also make sense in reverse, embedding not so much.)

Module authors should be able to disable generating the API docs if they want to provide 1. only (still being able to embed parts of 2.).

As for security, here is a speculation: In order to make this completely safe in a fairly general case including BEGIN block, I think - let me know if I'm wrong - that we could generate the AST of a source (a rakumod or rakudoc file) inside a secure container, serialise the AST.rakudoc, then render from the deserialised AST.rakudoc.

I believe this will work and is a quick solution freeing us of the need to define syntax for RakuDoc representations of Raku code. I still see value in having such a representation, as the intermediate format is then plain RakuDoc. This incentivizes creating tools that process RakuDoc exclusively. Also the intermediate format is then human readable and writable.

[finanalyst]

We are dealing with two separate sets of documentation that follow a different structure.

1. The non-declarator-block RakuDoc parts follow a top to bottom ordering in a file and structured in a flat list of file names. RakuDoc provides flexible tools to introduce ordering via automatic or manually created TOCs, indexes and links.

2. The deck (declarator-block) RakuDoc parts follow the hierarchical structure of the code. The sum of all decks form a single tree of `package / namespace -> class / role -> method / sub`.

This does not seem to be a good description. All sources (by which I mean the files with .rakumod or .rakudoc extensions) are parsed into ASTs. The AST describes the structure.

Sources are themselves gathered into directories. Hence the name 'Collection' for how to process them all. The RakuDoc specification makes no mention of the directories, and the Raku documentation suite is not a flat list of filenames.

The sum of all decks is a structure, which we can create as we run through the AST, gathering decks from branches. The same is true of the ToC, Footnotes, Semantic, and Index structures, where information is gathered from =head, N<>, X<> and =NAME_FOLLOWING_SEMANTIC_RULES.

How these data are then rendered is quite arbitrary.

What I suggested above would be easily added to the existing rakuast-rakudoc-render by adding a handler for Declarator blocks, and a template for rendering them.

[patrickbkr]

This does not seem to be a good description. All sources (by which I mean the files with .rakumod or .rakudoc extensions) are parsed into ASTs. The AST describes the structure.

I think there is a misunderstanding here. I was talking about the abstract logical structure of free form documentation and decks, not how the different elements are technically layed out in our AST representation.

Maybe I also misunderstood your intent from the following:

At the moment, it would be useful for me at least, if we could focus on how to manage the AST that is now being generated.
...
I don't have much idea about how this might look.
...
Suppose that all declarations are gathered together into a Table, perhaps called DECTABLE, is treated as an implicitly hidden semantic block. Then any where inside a Rakudoc source we could have
=place semantic:DECTABLE :caption<Declaration blocks>, but for a default, we place it at the end.

I gathered that you stated that decks fall out of the usual flow of the documentation and are somehow separate. And you are asking for feedback how to present and especially order/place decks in relation to non-deck content. Did I misread your post?

How these data are then rendered is quite arbitrary.

What I suggested above would be easily added to the existing rakuast-rakudoc-render by adding a handler for Declarator blocks, and a template for rendering them.

Simple solutions are good! I'm just unsure if the outcome is as useful as it can be.

[finanalyst]

Hi Patrick, After posting the referenced response, I worried my tone was too rough. Thank you for responding to the content not the tone. Some personal thoughts of mine. There is a need to document Raku software in a way that is easy for a coder to implement as she is coding. Declaration blocks are a good way to do this, together with other elements of Rakudoc v2 Despite having great tools, they are insufficiently used IMHO in general in the Raku community. Some recommendations would go a long way to change culture. Also a good rendering into Html and it's use in Raku land would encourage coders to make the effort needed to improve their documentation. Speaking about documentation of software intermingling decks and blocks, I don't know how this should or could look in terms of HTML. If you or other contributors to this discussion could give examples of good rendered (into HTML) documentation, we could draw some generic conclusions. I think it would be easy to render raku AST into any documentation format. Also, given we have lots of information in an AST and the ability to deparse code, we should be able to separate out methods (etc) and signatures to include them too. What it seems to me we agree on is there are two types/goals of documentation. This is I think a useful conversation. Have a good day. Richard

…

On Thu, 9 Jan 2025, 12:25 Patrick Böker, ***@***.***> wrote: This does not seem to be a good description. All sources (by which I mean the files with .rakumod or .rakudoc extensions) are parsed into ASTs. The AST describes the structure. I think there is a misunderstanding here. I was talking about the abstract logical structure of free form documentation and decks, not how the different elements are technically layed out in our AST representation. Maybe I also misunderstood your intent from the following: At the moment, it would be useful for me at least, if we could focus on how to manage the AST that is now being generated. ... I don't have much idea about how this might look. ... Suppose that all declarations are gathered together into a Table, perhaps called DECTABLE, is treated as an implicitly hidden semantic block. Then any where inside a Rakudoc source we could have =place semantic:DECTABLE :caption<Declaration blocks>, but for a default, we place it at the end. I gathered that you stated that decks fall out of the usual flow of the documentation and are somehow separate. And you are asking for feedback how to present and especially order/place decks in relation to non-deck content. Did I misread your post? How these data are then rendered is quite arbitrary. What I suggested above would be easily added to the existing rakuast-rakudoc-render by adding a handler for Declarator blocks, and a template for rendering them. Simple solutions are good! I'm just unsure if the outcome is as useful as it can be. — Reply to this email directly, view it on GitHub <#439 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACYZHBXQKBLXIHKPA2MIZ32JZTCRAVCNFSM6AAAAABOLVSHH6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKOBQGAZDSNZYGA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[patrickbkr]

Some personal thoughts of mine. There is a need to document Raku software in a way that is easy for a coder to implement as she is coding. Declaration blocks are a good way to do this, together with other elements of Rakudoc v2 Despite having great tools, they are insufficiently used IMHO in general in the Raku community. Some recommendations would go a long way to change culture. Also a good rendering into Html and it's use in Raku land would encourage coders to make the effort needed to improve their documentation.

I holeheartedly agree. We have the opportunity to change the tides in how the Raku ecosystem is documented when we manage to introduce ways to view documentation.

Speaking about documentation of software intermingling decks and blocks, I don't know how this should or could look in terms of HTML. If you or other contributors to this discussion could give examples of good rendered (into HTML) documentation, we could draw some generic conclusions.

I think I either completely fail to communicate my vision for how a good documentation could look, or I'm deluding myself thinking I have a good idea, while it's not so good after all. :-P

Did you notice the Sphinx and Doxygen links I posted a few posts above? I guess the free form documentation that Spinx documentation page resembles should be no news to you, given that our Raku documentation follows essentially the same style. (free form text, structured via paragraphs, headings, TOCs, indexes, links, ...) The Doxygen link points to a documentation page generated by a tool called Doxygen. It scans C++ project sources and extracts all classes and methods and the associated documentation. Code documentation works essentially the same as decks work in Raku, one prefixes a class or method with a comment that is marked up so Doxygen can find and process it. It then generates one page for each class. Such a page then presents the public API of that class. I.e. it lists all publicly accessible members of that class and their documentation. The website as a hole captures the entire public API of that project. The content is structured following namespaces, classes and methods.

My idea was to separate decks from non-decks entirely and generate two separate sets of documentation pages. One resembling free form documentation and another resembling a Doxygen generated API doc. By allowing linking and embedding from one set into the other module authors can connect the two sets of documentation where it makes sense.

I only now realize, that the official Raku doc also already mostly follows such a structure. 🤦🏽 There is a Reference section containing free form documentation. And there also is a Type section containing API documentation for classes and similar. The only essential difference is, that the Raku documentation is not using decks to write the API documentation but recreating the structure using free form documentation. (I recall that that was a deliberate decision as to not couple the Raku documentation to the Rakudo implementation.)

[finanalyst]

I tried looking at the links you sent me but there was so much stuff, I couldn't understand what to look at. Also I have spent so much time on the Rakudoc specification and the Raku docs website, I know far more than is healthy about those things without having a broader view. When you sent me the links to documentation systems, were you suggesting you like the website themselves or the products they create? If the latter, I really need to see an example. I'm nearly ending work on a first draft of a new Raku documentation site. It's close to the current one, but with some extra functionality. That's taking time at the moment. Richard

…

On Thu, 9 Jan 2025, 14:16 Patrick Böker, ***@***.***> wrote: Some personal thoughts of mine. There is a need to document Raku software in a way that is easy for a coder to implement as she is coding. Declaration blocks are a good way to do this, together with other elements of Rakudoc v2 Despite having great tools, they are insufficiently used IMHO in general in the Raku community. Some recommendations would go a long way to change culture. Also a good rendering into Html and it's use in Raku land would encourage coders to make the effort needed to improve their documentation. I holeheartedly agree. We have the opportunity to change the tides in how the Raku ecosystem is documented when we manage to introduce ways to view documentation. Speaking about documentation of software intermingling decks and blocks, I don't know how this should or could look in terms of HTML. If you or other contributors to this discussion could give examples of good rendered (into HTML) documentation, we could draw some generic conclusions. I think I either completely fail to communicate my vision for how a good documentation could look, or I'm deluding myself thinking I have a good idea, while it's not so good after all. :-P Did you notice the Sphinx <https://www.sphinx-doc.org/en/master/index.html> and Doxygen <https://api.kde.org/frameworks/ktexteditor/html/classKTextEditor_1_1Cursor.html> links I posted a few posts above? I guess the free form documentation that Spinx documentation page resembles should be no news to you, given that our Raku documentation follows essentially the same style. (free form text, structured via paragraphs, headings, TOCs, indexes, links, ...) The Doxygen link points to a documentation page generated by a tool called Doxygen. It scans C++ project sources and extracts all classes and methods and the associated documentation. Code documentation works essentially the same as decks work in Raku, one prefixes a class or method with a comment that is marked up so Doxygen can find and process it. It then generates one page for each class. Such a page then presents the public API of that class. I.e. it lists all publicly accessible members of that class and their documentation. The website as a hole captures the entire public API of that project. The content is structured following namespaces, classes and methods. My idea was to separate decks from non-decks entirely and generate two separate sets of documentation pages. One resembling free form documentation and another resembling a Doxygen generated API doc. By allowing linking and embedding from one set into the other module authors can connect the two sets of documentation where it makes sense. I only now realize, that the official Raku doc also already mostly follows such a structure. 🤦🏽 There is a Reference <https://docs.raku.org/reference> section containing free form documentation. And there also is a Type <https://docs.raku.org/type/Map> section containing API documentation for classes and similar. The only essential difference is, that the Raku documentation is not using decks to write the API documentation but recreating the structure using free form documentation. (I recall that that was a deliberate decision as to not couple the Raku documentation to the Rakudo implementation.) — Reply to this email directly, view it on GitHub <#439 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACYZHH4DVPCQYBPBK5DY5L2J2ACRAVCNFSM6AAAAABOLVSHH6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKOBQGMZDMMBTGI> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[patrickbkr]

When you sent me the links to documentation systems, were you suggesting you like the website themselves or the products they create? If the latter, I really need to see an example.

I don't have much interest in Doxygen, JavaDoc or Sphinx (that's the tooling of other languages, we're building our own tooling right now anyways). I only provided the links as an example of how an API doc generated from commented code can look. The Doxygen link points to the documentation of the Cursor class that's part of a text editor (KTextEditor) that's part of the KDE desktop environment. So that page is documenting a single class. The actual content is irrelevant, but the layout is interesting. It follows this format:

• Name of the class
• the #include statement necessary to use it (the Raku equivalent is use)
• A tabular list of all public methods
• A free form introduction text
• An exhaustive list of available methods, starting with the constructors:
  • The method signature
  • What does the method do?
  • What is the meaning of each argument? (they call it Parameters)
  • Where in the code the method defined?

I did realize, that the official Raku docs essentially show case the same documentation style. So there is not much need to pull in external examples.

Let's look at the documentation of the Map class. It contains these parts:

• Name and parent classes / roles
• A free form introduction text
  • What is a Map?
  • How do I create a Map?
  • How do I use a Map?
• An exhaustive list of available methods:
  • The method signature
  • What does the method do?
  • What is the meaning of each argument?
  • Sometimes an example

Do you think it would help if I'd create a mockup of an API documentation page for a Raku class?

I'm nearly ending work on a first draft of a new Raku documentation site. It's close to the current one, but with some extra functionality. That's taking time at the moment.

Nice! Don't feel pressed to spend too much time on this discussion. I guess we're some way off before work can start for real on this.

[dontlaugh]

I'm nearly ending work on a first draft of a new Raku documentation site.

@finanalyst You are referring to this, right? Raku/doc-website#429

[finanalyst]

@dontlaugh yes. Although with hindsight, I maybe should refer to this project differently. And also, 'nearly ending' is not as close as I imagine.