Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prototype for span-based ASTs #5

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft

Prototype for span-based ASTs #5

wants to merge 1 commit into from

Conversation

Hirevo
Copy link
Owner

@Hirevo Hirevo commented Jun 16, 2020

This PR experiments with refactoring the lexer and the parser to entirely rely on spans, instead of directly constructing values.
This makes the construction of the AST a bit more efficient, since it reduces the number of allocations needed.

The interpreter then takes the AST and can resolve spans to string slices (an efficient non-owning view into a string) when needed, by keeping around the source code for each class definitions.

It opens the door for better error-reporting, allowing to show proper stack-traces to the user (although not done here yet and probably not within the scope of this PR).
But it also means that things like integer literals need to be parsed into actual integers every time we evaluate one.

I haven't done any measurements yet on the performance delta with the original implementation, so I cannot say if this is an actual improvement over it.

The implementation should already be feature-complete and in parity with the features of the original interpreter.

@Hirevo Hirevo added C-refactor Category: Refactors M-lexer Module: Lexer M-parser Module: Parser M-interpreter Module: Interpreter P-medium Priority: Medium labels Jun 16, 2020
@Hirevo Hirevo self-assigned this Jun 16, 2020
@smarr
Copy link
Contributor

smarr commented Jun 16, 2020

But it also means that things like integer literals need to be parsed into actual integers every time we evaluate one.

Hm, could you simply cache the results after the first evaluation?
Re-parsing things like integers and string seems like something potentially expensive, for instance when done in a loop:

| cnt |
1 to: 1000 do: [:i |
  cnt := cnt + 1. "Is this 1 always reparsed?" ].

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-refactor Category: Refactors M-interpreter Module: Interpreter M-lexer Module: Lexer M-parser Module: Parser P-medium Priority: Medium
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants