Allow options to override default node types and mark types

[dylans]

Have you considered allowing a second parameter to support custom node types and mark types within an AST?

I'm trying to decide between remark-slate and remark-slate-transformer. This package handles a far more comprehensive set of markdown but would require an additional step of parsing from Slate to the AST we store which uses custom node types.

I'm happy to contribute a PR if this is a feature you'd like to add, just respond here or find me on the Slate slack group.

[inokawa]

Hi, @dylans.

Yes. I thought it may be needed if someone use this seriously. Thank you!

This plugin trusts mdast data structure (the type definition in this plugin is bit outdated but almost the same).
Basically this converts between a mdast node and a slate node with each functions and does special conversion for emphasis/strong/delete/inlineCode to fill the gap of mdast and slate, so overriding the functions should work I guess.

[dylans]

Thanks @inokawa , I'll put something together soon and get your feedback.

[dylans]

ok, I finally have something as a starting point for discussion (the code itself is far from efficient and should at least illustrate the concept): master...dylans:ast-custom-tags (obviously it would be much better to decide on a pattern to override the default ast type with the parameters passed in more efficiently than what I've done here, but it would also be ideal to preserve roughly the same defaults structure people might use in configuring slate or slate-plugins.

Please have a look, let me know if you have thoughts or feedback. I'm also happy to discuss it in real-time if that's easier. :)

[dylans]

It's also an entirely reasonable thing to say that maybe this should instead be a separate ast step, e.g. slate ast -> customized slate ast.

[inokawa]

Thank you @dylans.

Please point out if you think I misunderstood. If so that is because of my English skills...

This lib handles mdast ast <-> slate ast, so you mean that the new option would be like slate ast -> customized slate ast right?
I think the way to pass a map is nice but I felt direct conversion like mdast ast -> customized slate ast may be better.

The mdast ast suports all markdown syntax and it is documented, and there are no default slate structure because it is customizable.
So the users may be easier to write option referencing mdast document than searching my implicit slate definition...

I come up with passing (mdast node) => slate node functions may work but not sure. Something like this? (This is just an uncertain idea)
I may try to implement this if I have time. What do you think about?

use(remarkToSlate, {
  heading: (mdast, createChildren) => ({
    type: mdast.type,
    [mdast.depth]: true,
    children: createChildren(mdast.children),
  }),
  paragraph: (mdast, createChildren) => ({
  ...
  })
});

[dylans]

Thanks @inokawa , I agree. I think the more I tried to force things the way I did, the more I didn’t like the approach I took. slate-plugins and remark-slate follow the approach I proposed but it really is cumbersome to manage so I think I like your suggested approach. I’ll think about it more in the morning and let you know if I have further thoughts.

[siliconeidolon]

I also prefer this library to remark-slate and it'd be great to have the option to add additional types, eg: underline. I know it isn't part of the spec, but it is a common feature of WYSIWYG editors.

[inokawa]

Hi, @siliconeidolon .
Some decorations like underline is not a common mdast spec but you can use them with HTML in markdown.
Texts like <ul>foo</ul> will be parsed into type: html node, and you can get raw HTML and can render it in Slate with some way like https://docs.slatejs.org/concepts/09-serializing#deserializing.

Add new feature to handle unknown nodes to this plugin is possible, but probably harder to achieve.

[siliconeidolon]

@inokawa Thanks for responding.

So, I should first process my slate content to look for my custom mark (eg: underline) and then run slateToRemark?
Something like:

const preprocess = (contentObj: Node) => {

  if (contentObj.text) {
    if (contentObj.underline) {
      return { ...contentObj, type: 'html', text: `<u>${contentObj.text}</u>` }
    } else {
      return contentObj
    }
  }

  if (contentObj.children) {
    const updatedChildren = contentObj.children.map(child => preprocess(child))
    return { ...contentObj, children: updatedChildren }
  }

  return contentObj
}

const processor =
    unified()
      .use(slateToRemark)
      .use(stringify)

const serializedContent = processor.stringify(
    processor.runSync({
      type:     'root',
      children: preprocessedSlateContent,
    }))

preprocessedSlateContent:

{
  "type": "paragraph",
  "children": [
    {
      "text": "<u>The</u>",
      "strong": true,
      "underline": true,
      "type": "html"
    },
    {
      "text": " modern day Aberdeen Football Club was created in a merger between three clubs: Aberdeen, Victoria United, and Orion."
    }
  ]
}

serializedContent:
**\<u>The\</u>** modern day Aberdeen Football Club was created in a merger between three clubs: Aberdeen, Victoria United, and Orion.

This almost works, but I wonder where the escape characters are coming from?

[inokawa]

The result looks not correct because underline in my demo looks not escaped by this lib.

I guess it's caused by the bold wrapping HTML.
I didn't expect to use HTML with decorations so my lib or remark may not work as expected about them.

[inokawa]

I added overrides options in #89 .

This is initial support of custom AST and may have some problems.
Overriding may not work enough on PhrasingContent in mdast / Text in slate for now.

[siliconeidolon]

Thanks for adding this. It could potentially make my de/serialisation functions a lot simpler!

[vavilov2212]

I had the same problem with back slashes. If someone is looking for a solution, here's mine.

  const value = [{ 
      type: 'paragraph',
      children: [
        { text: 'This text is underlined.', underline: true },
      ],
    }];

  const processor = unified()
    .use(slateToRemark, {
      overrides: {
        paragraph: (node, next) => {
          const children = node.children.map(child => {
            if (child.text && child.underline) {
              const { text, ...modifiedChild } = child;
              return {
                ...modifiedChild,
                type: 'html',
                children: [ { text: `<u>${child.text}</u>` }]
              }
            }
            return child;
          });
          return ({
            type: "paragraph",
            children: next(children),
          });
        },
      },})
      .use(stringify);

  const ast = processor.runSync({
    type: "root",
    children: value,
  });
  const mdText = processor.stringify(ast);

  console.log(mdText);
// <u>This text is underlined.</u>

[janaka]

@vavilov2212 Hey, does that above work for you in the latest version of the plugin? I'm having issues with slateToRemark. remarkToSlate works fine.

Update: figured it out finally by digging into the slateToRemark source. My overrides for slateToRemark can't be exactly the same as remarkToSlate because I'm overriding the type. e.g. 'paragraph' -> 'p'. I need to map in opposite directions for serialize and deserialize. I missed this in the README example.

Why am I doing this? because I'm using Slate with Plate and for some reason, Plate uses different names for the types (which is pretty annoying).

So for example:

 .use(slateToRemark, {
    overrides: {
      p: (node: any, next: any) => ({
        type: 'paragraph',
        depth: node.depth,
        // You have to call next if the node has children
        children: next(node.children),
      })
    }
  })

.use(remarkToSlate, {
    // If you use TypeScript, install `@types/mdast` for autocomplete.
    overrides: {
    paragraph: (node: any, next:any) => ({
        type: 'p',
        depth: node.depth,
        // You have to call next if the node has children
        children: next(node.children),
      })
    }
  })

[janaka]

Is overriding nodes like inlineCode and strong supported? if so can you give an example. I need to go from inlineCode=true to code=true

[inokawa]

@janaka
Overriding PhrasingContent in mdast or Text in slate may not work in some cases as describe in #31 (comment)
We can fix them if you have reproduction.