MarkOrg: .morg and literate documentation

[drewc]

MarkOrg: GitHub Flavored Markdown Org

Documentation, and specifically reference documentation, needs to be so tightly integrated with Gerbil and the Gerbil Experience that almost everything meta to do with an identifier is available at the developers fingertips with almost no effort on their part.

It also needs to be jotted down in an easy to read/write and widely used format that can also be collected in a general anthology.

Code and Reference are separated in the source tree but united with an IDE. The source of that documentation should be literate and mostly in the source tree so a developer can find and read and update the documentation as easy as a .ss file but at the same time should not be a part of the source files.

In short we need a parser, object, and printer for a Markdown that has some Org DNA in it. Or, if you're reading this on github as a web page imagine something like this being slightly more useful:

::: tip
#+module: :std/markup/md
#+id: read-markdown
:::

and if you view the source of this code block:

> (+ 31 11)
42

... it has something hidden.

```scheme :noweb-ref morg-test
 > (+ 31 11)
 42
```

Motivation: @doc, # source, and literate tests.

At the REPL or within the IDE a way to describe a symbol or view the source code is a big plus. Interacting with the source code from within the IDE or the reference documentation is also an added bonus. Generating the reference website and friends from the documentation source is well needed. And having some test cases be automatically generated (AKA "tangled") from the documentation ensures that certain things agree with one another.

Markdown: GH + VP

Markdown is a lightweight markup language for creating formatted text using a plain-text editor. John Gruber created Markdown in 2004 as an easy-to-read markup language. Markdown is widely used for blogging and instant messaging, and also used elsewhere in online forums, collaborative software, documentation pages, and readme files.

Github uses it and our main repo is github.

• GH Markdown : : https://github.github.com/gfm/

VuePress containers

We currently use VuePress to generate our website. In particular we use the containers for useful tips.

::: tip
This is a tip

:::

::: warning
This is a warning with an image

![Warning image](./assets/warning.png)
:::

::: danger That is my middle name
This is a dangerous warning with unicode emoji 🚨
:::

Org Mode

Org Mode /ˈɔːrɡ moʊd/) is a mode for document editing, formatting, and organizing within the free software text editor GNU Emacs and its derivatives, designed for notes, planning, and authoring. The name is used to encompass plain text files ("org files") that include simple marks to indicate levels of a hierarchy (such as the outline of an essay, a topic list with subtopics, nested computer code, etc.), and an editor with functions that can read the markup and manipulate hierarchy elements (expand/hide elements, move blocks of elements, check off to-do list items, etc.).

Org Mode was created by Carsten Dominik in 2003, originally to organize his own life and work.

Now github also has org-mode but it is not as widespread as markdown, is mostly an Emacs based ecosystem, and @vyzo (BDFL) doesn't use it.

On the other hand it has #+KEYWORDS: val ... which do not conflict with github's header syntax and Noweb syntax whereby you can play with a "collection of one or more source code blocks sharing the same :noweb-ref’" header argument.

So we can use keywords.

#+id: name
#+module: :foo/bar/baz

Also a :noweb-ref header argument.

```scheme :noweb-ref baz-test
> (answer? 42)
#t
````

Both of which can be gotten away with on github.

Exporting (aka weave)

I'll immediately do an md parser that exports to html so we can get rid of netlifiytfylypressthing. We've got TAGS already so even without the #+id: and #+module: keywords (#+namespace:? #+package: ? #+lang:?) I can match headers and files to the source and have a # source link on the HTML side.

Possibly even a typesig autodoc! W00t!

@doc or @describe even.

It would be nice to have apropos and friends be linked with "external" docs. We already will have # source so this is just the other direction more or less.

Tangling tests

Thought I agree that source files (.ss ) in the tree should not be generated from .morg files for the most part, tests can and should be!

I've already done so using org and a > test result type syntax so making it a part of the main tree itself would be a good thing. It encourages both writing tests, writing docs, and if you're a literate programmer such as myself, almost the entire process.

So the test file is generated from something like:

(defsyntax (test-inline stx)
      (syntax-case stx (>)
        ((_ test-case: name rest ...)
         #'(test-case name (test-inline rest ...)))
        ((_ > form > rest ...)
         #'(begin (when std/test#*test-verbose* (displayln "... " 'form)) form (test-inline > rest ...)))
        ((_ > test result rest ...)
         #'(begin (check test => 'result) (test-inline rest ...)))
        ((_) #!void)))
    
(def parsec-test 
  (test-suite "Test :std/parsec"
   (test-inline
    test-case: "Char Reader tests"
    <<char-reader-test>>)) ...)

Combined with blocks in the "documentation" like:

 ```scheme :noweb-ref char-reader-test
 > (char=? #\4 (P.run (P.read-char) "42"))
 #t
 ```

... means a bunch of good things happen almost by themselves

Web REPL/IDE/EDITOR and the gerbil.js friend

Gerbil's are processes and on top of them could be a good webapp and I've done gerbil using the Universal Backend of Gambit many times so I know how close we are. With distributed ensembles and cloud potential a web app (which can be read as "phone app" as well) will be a great step.

Because we'll be making our own "html" from the "markorgs" making that into an application is not that hard and the intended next step.

TODO and thoughts?

I'll do the parser and unite all of our docs and start making it all a part of the IDE. Where we'll go in anyone's guess 👯‍♀️

[drewc]

::: tip Examples: could also be the "test this"? Working on a the parser and discussing with myself :)

[vyzo]

related discussion in gambit: gambit/gambit#920

[drewc]

Lol. I hate Texinfo slightly less than I hate XML. Neither are good documentation formats for contribution so I suspect "parse tex to make a markdown to publish" and "treat all gambit docs as external and in a bad format" will likely be my take on that. Contribution is way more important than mythical standardization of poor formats and I've only used my printer once, for one page, so typesetting is not on my radar. So happy the Gerbil is for real use and not constrained in that way :) I won't comment on that thread as my criticism is not very useful as I prefer reading source trees over reading books when it comes to both using and contributing.

…

On Sun, Sep 15, 2024 at 8:06 AM vyzo ***@***.***> wrote: related discussion in gambit: gambit/gambit#920 <gambit/gambit#920> — Reply to this email directly, view it on GitHub <#1269 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADVTXNRGSJW2L5FGDPS5RDZWWPAPAVCNFSM6AAAAABN3JX4HGVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTANRVGI2TENQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>