Get in line with CommonMark

[cebe]

https://github.com/jgm/CommonMark

[samdark]

It's not CommonMark http://commonmark.org/

[philsturgeon]

So, what is the plan here folks?

[cebe]

the spec is still kind of unstable, I don't think it is worth working on an implementation when I have to change it every week... I am following the changes at CommonMark and might start implementing in a few weeks, not sure when I find time for it.

[philsturgeon]

I know that moving target specs are not ideal, but I am not sure CommonMark can be considered unstable. They are not flip flopping on implementation results, just slowly adding more rules to it.

It could be cool to get the test suite from the CommonMark and run a branch against it, so we can see how this implementation compares to the results expected by the spec. Parsedown did a great job of this.

The League of Extraordinary Packages is going to take over this PHP implementation that is currently - and will continue to be - fully spec compliant. It's made it onto the list of implementations on the CommonMark wiki, even though it is considerably slower than the non-CommonMark implementations.

We could find a way to work together, if you aren't as eager to run and jump into the world of CommonMark just yet. We can definitely look into sharing a test-suite as the more examples the better.

Another benefit here is there is no legacy to worry about, where a CommonMark rule might conflict with what an existing implementation result. If BC gets in the way, we'll have the advantage of just going for it. If you could help with speed while we focus on compliance, then we'll have a winner of a product.

[cebe]

While it looks like only adding more and more small rules, complying to these rules sometimes needs big changes in parsing strategy. I don't have a good feeling about how much effort it would be or how good my parser fits to what CommonMark does, I can only guess and the devil is always in the details here. It is different if you use a parser that follows CommonMark, your output may not change much as you may not have many edge cases in your texts. It is different for the implementation of a parser as a small change in the specs may result in a big change in parsing strategy.

I have just run the test suite, (and fixed it to work properly :) commonmark/commonmark-spec#229):

With Markdown class:
243 passed, 266 failed, 0 errored, 0 skipped
With GithubMarkdown class:
259 passed, 250 failed, 0 errored, 0 skipped

have not looked into the details yet but it looks like simple stuff right now to get that numbers to something more like 80% or so.

We could find a way to work together, if you aren't as eager to run and jump into the world of CommonMark just yet. We can definitely look into sharing a test-suite as the more examples the better. [...]
If you could help with speed while we focus on compliance, then we'll have a winner of a product.

Not sure I understand, do you invite me to contribute to colinodell/commonmark-php or you want my implementation to be in the The League of Extraordinary Packages?

[philsturgeon]

My basic plan - as such as it is a plan - is to get a PHP PECL made up.

I was looking into using the cmake library directly with a PHP equivalent of ctypes or ffi but luckily a core PHP developer is looking into this as well:

https://twitter.com/auroraeosrose/status/539480522259718144

If she can make a slick PECL for this that we can suggest as an option, it will be much quicker than any user land PHP implementation can be. Then PHP Markdown implementations like your own can just be a thin wrapper around this if they do not wish to create their own userland implementation.

league/markdown is going to be the new colinodell/commonmark-php which will act as a userland implementation of CommonMark, but also support the PECL. We will play keep-up with the CommonMark specification as it moves and make this a 100% compliant library via userland and/or PECL.

If you can see anywhere in there to help us, like perhaps helping us improve the speed of our CommonMark userland implementation, and add in extensibility, then we would have one hell of a sweet solution.

[cebe]

Yeah, having a pecl implementation for pure and simple parsing is a nice idea. A userland implementation is useful when you want to extend the markdown language for example adding things like user mentions, issue references or emoij like they are on github in your own application.

Also the goal of my lib is to provide a way to manipulate or extract information from markdown based on an abstract syntax tree. This may only be possible in a PECL extension with cmark which has this option, not sure about other implementations.

Working on a draft of a CommonMark implementation right now to see how it goes...

Not sure if I am going to dig into colinodell/commonmark-php, maintaining one markdown implementation is enough work for me and mine is the more general approach as it is not limited to CommonMark.

[cebe]

Started working on a CommonMark implementation to see how it would turn out. It is really quite different from how I have seen and parsed markdown before.

https://github.com/cebe/markdown/compare/common-mark

Current test results: 303 passed, 206 failed, 0 errored, 0 skipped.

[philsturgeon]

Nope, not asking you to start on colinodell/commonmark-php. We've already made a start on league/markdown and it'll keep on going.

The PECL might help you out when you want a CommonMark driver, and our userland package might help you out if you want a CommonMark driver that doesn't need a PECL.

If, in the future, you feel like contributing to league/markdown to help speed it up, once CommonMark supports directives or extensions (and has things like tables covered) then you might not find the need to keep this one going so hard.

I'm just saying we should try and find some ways to work together and share ideas, as the days of wildly different Markdown implementations need to be a long way behind us. :)

[cebe]

My focus is to not only have a parser that translates from MD->HTML but also an implementation that is extensible and allows to work on the markdown syntax tree like for example on http://www.yiiframework.com/doc-2.0/guide-index.html the navigation is built automatically from the markdown that was parsed from the index document.

So this lib has its use cases even if there is a PECL implementation and a simple userland parser.
Also IMO there can not be one common markdown spec but that would take a long discussion to explain in detail :)

I have watched the repo, will at least take part in discussions.

[philsturgeon]

Good man, thank you! 

-- 
Phil

On December 1, 2014 at 11:48:48 PM, Carsten Brandt ([email protected]) wrote:

My focus is to not only have a parser that translates from MD->HTML but also an implementation that is extensible and allows to work on the markdown syntax tree like for example on http://www.yiiframework.com/doc-2.0/guide-index.html the navigation is built automatically from the markdown that was parsed from the index document.

So this lib has its use cases even if there is a PECL implementation and a simple userland parser.
Also IMO there can not be one common markdown spec but that would take a long discussion to explain in detail :)

I have watched the repo, will at least take part in discussions.

—
Reply to this email directly or view it on GitHub.

[maximal]

+1 here

Commonmark as an open and unambiguous markwown standard is a really good idea.