Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

implementation of an AST -> SCFG transformer #114

Merged
merged 22 commits into from
Apr 29, 2024
Merged

implementation of an AST -> SCFG transformer #114

merged 22 commits into from
Apr 29, 2024

Commits on Apr 19, 2024

  1. implementation of an AST -> SCFG tranformer

    AST: Python Abstract Synatx Tree
    (S)CFG: (Structured) Control Flow Graph
    
    Items to note for reviewers:
    
    * Most of changes are in the `ast_transforms.py` module which contains
      the meat of the implementation.
    * Temporary data structures are created for a pure AST based CFG
      (`ASTCFG`) with blocks that are writable. This made it much easier to
      implement the builder pattern. They can easily be converted to a SCFG
      using the `ASTCFG.to_SCFG()` function.
    * The `AST2SCFGTranformer` is stateful and is populated with
      housekeeping data structures post transform.
    * The tests are all in `test_ast_transforms.py`. They are of the usual
      format: Python function as input and serialized graph to compare to.
    * The tests also check the values of the unreachable and empty nodes
      that have been pruned from the CFG.
    * The for-loop decomposition is a bit involved, hence there is an
      extensive docstring to describe what it does.
    * Some minor changes had to be made to the `scfg.py` module to support
      displaying "broken" SCFGs during development. This makes it easier to
      visually debug the graph upon making programming mistakes.
    * The modules `basic_block.py` and `rendering.py` were modified to
      support construction and visualization of the `PythonASTBlock`, which
      is a new block type for the SCFG that contains a block derived from
      Python AST.
    * The central entrypoint is: `AST2CFG(function) -> SCFG `. This will
      turn a Python function into an SCFG.
    * A stub has been added for the invserse, SCFG -> AST transformer.
    
    Example:
    
    ```python
    from numba_rvsdg import AST2SCFG
    def fun():
        ...
    scfg = AST2SCFG(fun)
    scfg.render()
    scfg.restructure()
    scfg.render()
    ```
    esc committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    d9a33f1 View commit details
    Browse the repository at this point in the history
  2. cleanup imports

    As title
    esc committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    c1ec0cc View commit details
    Browse the repository at this point in the history
  3. tweaking comments

    As title
    esc committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    7ae7668 View commit details
    Browse the repository at this point in the history
  4. add to_dict for WritableASTBlock

    As title
    esc committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    377d161 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    a84bdd8 View commit details
    Browse the repository at this point in the history
  6. remove break and continue nodes

    This reduces the graph even more, since blocks with a single break or continue become empty blocks and are pruned
    esc committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    caa5ea2 View commit details
    Browse the repository at this point in the history

Commits on Apr 20, 2024

  1. tweak docstrings and comments

    As title
    esc committed Apr 20, 2024
    Configuration menu
    Copy the full SHA
    80ae78e View commit details
    Browse the repository at this point in the history
  2. bugfix: set jump targets after recursion

    The jump_targets for the else-branch of the for-loop must be set after
    recursion, not before. Otherwise nested flow-control constructs in the
    else-branch will fail to codegen correctly, leading to a broken CFG with
    blocks that point to non-existing blocks. The test exposes this and
    demonstrates the fix is valid.
    esc committed Apr 20, 2024
    Configuration menu
    Copy the full SHA
    ca8fdca View commit details
    Browse the repository at this point in the history

Commits on Apr 24, 2024

  1. bugfix: continue from nested else

    This test exposes a bug in a missing seal for using continue in the
    nested for-else clause. Specifically, one needs to have an else-clause
    with a series of linear statements and a continue to hit this. If you
    try to break, the break index will be the same as the exit index for the
    seal_inside_of_loop as the default_index is also the exit_index. So you
    can only hit this with exit_index. In that case the block with the
    continue does not point to the loop header but to the incorrect exit
    instead.
    
    The test has been constructed such that a potential reconstruction would
    also fail execution with an incorrect return value.
    esc committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    a497881 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    dc64724 View commit details
    Browse the repository at this point in the history
  3. black comments

    As title
    esc committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    8d78bd3 View commit details
    Browse the repository at this point in the history

Commits on Apr 25, 2024

  1. Configuration menu
    Copy the full SHA
    8a119a4 View commit details
    Browse the repository at this point in the history
  2. black found a whitespace

    As title
    esc committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    4ab1cea View commit details
    Browse the repository at this point in the history
  3. type declaration for WritableASTBlock and ASTCFG

    As title
    esc committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    6579173 View commit details
    Browse the repository at this point in the history
  4. update code comment

    As title
    esc committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    b7908e5 View commit details
    Browse the repository at this point in the history
  5. implement while-else and simplify while handling

    As title
    esc committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    3563570 View commit details
    Browse the repository at this point in the history
  6. allow source as input for the transformer

    As title
    esc committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    7bb82cf View commit details
    Browse the repository at this point in the history
  7. fixup typing hints

    As title
    esc committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    040af33 View commit details
    Browse the repository at this point in the history
  8. make parsing/unparsing code a staticmethod

    As title
    esc committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    adfbc51 View commit details
    Browse the repository at this point in the history
  9. fix typo

    As title
    esc committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    d3137c5 View commit details
    Browse the repository at this point in the history

Commits on Apr 29, 2024

  1. test exercising transformer from string input

    As title
    esc committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    4b7d46b View commit details
    Browse the repository at this point in the history
  2. add ability to convert ast directly

    As title
    esc committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    41cc5e3 View commit details
    Browse the repository at this point in the history