From 7f408856b557d9fc5ed004f3d271dfa49a436b8c Mon Sep 17 00:00:00 2001 From: Ingvar Stepanyan Date: Tue, 11 Jul 2017 19:35:58 +0100 Subject: [PATCH] Updated the Quirks section --- js/README.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/js/README.md b/js/README.md index ceecd40..7b06d7c 100644 --- a/js/README.md +++ b/js/README.md @@ -48,7 +48,9 @@ TemplateToken ``` ### Quirks -Because the ECMAScript specification for `PunctuatorToken` (of which the `/` and `/=` symbols) and `RegexpToken` depends on a parser state to differentiate between the two, the lexer (to remain modular) uses different rules. Whenever `/` is encountered and the previous token is one of `(,=:[!&|?{};`, it returns a `RegexpToken`, otherwise it returns a `PunctuatorToken`. This is the same rule JSLint appears to use. +Because the ECMAScript specification for `PunctuatorToken` (of which the `/` and `/=` symbols) and `RegexpToken` depends on a parser state to differentiate between the two, the lexer (to remain modular) uses different rules. It aims to correctly disambiguate contexts and returns `RegexpToken` or `PunctuatorToken` where appropriate with only few exceptions which don't make much sense in runtime and so don't happen in a real-world code: function literal division (`x = function y(){} / z`) and object literal division (`x = {y:1} / z`). + +Another interesting case introduced by ES2015 is `yield` operator in function generators vs `yield` as an identifier in regular functions. This was done for backward compatibility, but is very hard to disambiguate correctly on a lexer level without essentially implementing entire parsing spec as a state machine and hurting performance, code readability and maintainability, so, instead, `yield` is just always assumed to be an operator. In combination with above paragraph, this means that, for example, `yield /x/i` will be always parsed as `yield`-ing regular expression and not as `yield` identifier divided by `x` and then `i`. There is no evidence though that this pattern occurs in any popular libraries. ### Examples ``` go