Support ES6 modules

[benderTheCrime]

events.js:141
      throw er; // Unhandled 'error' event
            ^
Error: Parsing file *: 'import' and 'export' may appear only with 'sourceType: module' (4:0)

I see in the README that ES6 code is supported, but I do no see any documentation supporting this. How would one use the documentation package to build documentation for an ES6 project?

[benderTheCrime]

This also does not appear to be specific to gulp-documentation or documentation, but this is still a relevant issue because I have not experienced it in any other package.

[anandthakker]

@benderTheCrime - Thanks for the report. Could you include the source file that causes the error above?

[benderTheCrime]

Sure can, it's throwing this error in the very first file I load using ES6 module imports:
https://github.com/benderTheCrime/angie/blob/master/src/Angular.js. The gulpfile I'm using is the one associated with that same project and is already registering ES6 files:
https://github.com/benderTheCrime/angie/blob/master/gulpfile.babel.js the documentation task is currently commented out). However, the command also fails with the very simplest variation of the tool from the command line: documentation src/Angular.js

[anandthakker]

Okay, I looked into this, and there are a couple of (deepish) issues:

1. The error above is thrown during dependency walking (module-deps, via node-detective) isn't handling ES6 modules correctly. Passing {ecmaVersion: 6, sourceType: 'module'} as options to acorn would solve this, but that'll require: (a) a patch to module-deps to allow those options, and (b) a patch to node-detective to pass the sourceType option through to acorn (it already passes ecmaVersion through).
2. ast-types doesn't have a definition for ExportNamedDeclaration (although it does have one for ExportDeclaration), so even after the module-deps issue is fixed, ES6 modules will blow up the parse() stream.

@benderTheCrime For now, one non-ideal workaround might be to use the polyglot: true option.

[anandthakker]

Okay, I looked into this, and there are a couple of (deepish) issues:

1. The error above is thrown during dependency walking (module-deps, via node-detective) isn't handling ES6 modules correctly. Passing {ecmaVersion: 6, sourceType: 'module'} as options to acorn would solve this, but that'll require: (a) a patch to module-deps to allow those options, and (b) a patch to node-detective to pass the sourceType option through to acorn (it already passes ecmaVersion through).
2. ast-types doesn't have a definition for ExportNamedDeclaration (although it does have one for ExportDeclaration), so even after the module-deps issue is fixed, ES6 modules will blow up the parse() stream.

@benderTheCrime For now, one non-ideal workaround might be to use the polyglot: true option.

[benderTheCrime]

Let me first try the polyglot: true option and see what my output looks like.

[benderTheCrime]

I don'y see any difference in the output with polyglot: true in my gulp task option arguments:

gulp.task('documentation', function() {
    gulp.src(src)
        .pipe(documentation({
            format: 'html',
            polyglot: true
        }))
        .pipe(gulp.dest('md-documentation'));
});

and I do not see documentation on passing the polyglot option to the CLI anywhere other than on the homepage: http://documentation.js.org/

Is there another way to do this?

[benderTheCrime]

Ah, it's --polyglot could you please add that to the documentation? Also, could the website be updated to reflect that the ES6 support is actually only available as a byproduct of using the language agnostic option? I've been vetting documentation tools and I spent a good deal of time on this tool (because it appears to have the best output) on what could have been avoided by some clarity. Otherwise, I think this is an awesome tool.

I'm also going to add an issue to the gulp project to make sure that the polyglot option is available.

[tmcw]

--polyglot is intended for non-JavaScript code: ES6 is supported but it looks like espree / ast-types are choking on the import/export declarations.

[anandthakker]

@tmcw @benderTheCrime Yeah, I should have been clearer: the polyglot idea was really just meant as a temporary workaround to force documentation not to do any parsing.

[tmcw]

The error above is thrown during dependency walking (module-deps, via node-detective) isn't handling ES6 modules correctly. Passing {ecmaVersion: 6, sourceType: 'module'} as options to acorn would solve this, but that'll require: (a) a patch to module-deps to allow those options, and (b) a patch to node-detective to pass the sourceType option through to acorn (it already passes ecmaVersion through).

@benderTheCrime do you have a package.json file with transforms defined? If you've got, for instance, babelify in your project to transpile ES6 to ES5, then documentation's module-deps instance will automatically use this transform. We shouldn't need to pass any options to acorn as above - module-deps transforms abstract away from any other language like ES6 or CoffeeScript and transform before resolving dependencies.

Given that not all projects that use documentation will use browserify or its "browserify" package key, this might be an area where we'll need to define transforms as passed-in options...

[benderTheCrime]

I'm using a gulpfile.babel.js. This immediately loads require('babel/register'); which transpiles all included ES6 modules when they are imported. This is a node/iojs project.

[tmcw]

On second thought, @anandthakker was totally right: we shouldn't need to babelify at that point, module-deps should permit passing options to acorn.

[anandthakker]

@tmcw I think passing that option would still be a superficial fix, because (I'm guessing) module-deps won't resolve ES6 module deps.

[benderTheCrime]

So what is the verdict here? Should I use polyglot until the espree / ast-types packages are upped?

[anandthakker]

@benderTheCrime Yep, I think that's the best bet in the short term. Not 100% sure on all the limitations you'll face, but one of them is that documentation won't automatically walk the dependency tree. (Since you're using gulp you could probably use that to rig up a way to run documentation on each individual file.)

[anandthakker]

Upstream: browserify/module-deps#93

Re: ast-types - looks like the newest version might have support, but haven't checked yet.

[fundon]

Is it fixed?

[tmcw]

As a general rule of thumb, if the issue is open, people are still working on the problem and it is not fixed yet.

[tmcw]

This is fixed in #163 which uses babel to transform code and generate an AST, and is covered by tests in cbff1d3 .

[thealjey]

@tmcw Is there any way I could already use this? Doesn't seem to work in 2.1.0-alpha2

[anandthakker]

@thealjey You could try installing straight off the github repo with npm install -g documentationjs/documentation#b173cf14a55dbed1ef6b214789c5218e1ab032a2

[thealjey]

@tmcw unfortunately this isn't really an option for me, but thanks anyway 😄
any plans on when this might land on npm?
sorry for bothering

[tmcw]

As soon as the 3.0.0 milestone is complete. We're moving fast, so it may be done by the end of the week.

[developit]

I am super pumped for this. What's left that is blocking 3.0.0?

[matthiasg]

this was confusing .. i have the same issue. Saw this as closed, but it isnt really published yet. thanks for the work though. BTW the --version command line parameter isnt really working.

[tmcw]

@matthiasg there isn't a --version flag. Opened #206 to add it.

the 3.0.0 milestone is everything that needs to be complete before it's released.

[tmcw]

Ah - going to rename the existing --version flag to --project-version for clarity.