[Chapel Merge] responsive compiler prototype: new AST classes

Notifications Commits
1

Branch: refs/heads/master
Revision: be4f2f7
Author: mppf
Log Message:

Merge pull request #17587 from mppf/new-ast-classes

responsive compiler prototype: new AST classes

This PR contains a few initial new AST classes for the compiler revamp
effort.

Split out from PR #17583, which is a start at the compiler revamp effort
described in the
1.24 release notes ongoing efforts.

The AST classes defined here only represent a subset of the language
(including { } blocks, trivial variable declarations like var a;, and
identifier statements like a;).

Here are some key points about the approach for defining the new AST
classes:

  1. As described in the Compiler Revamp slides, AST nodes have a unique
    ID. They can use pointers to refer to nodes within them, but can't use
    pointers to refer outside of them.
  2. The AST nodes are mostly-immutable. They are not entirely immutable
    because the process of parsing involves also assigning IDs to the
    nodes after they exist. However after the parsing is done they should
    not be modified. (This makes sense for the "query" part of the
    compiler; for later parts including optimization we will use a
    different AST/IR that is mutable).
  3. The AST nodes don't store types (in the sense of having a ->type
    field like the current AST nodes do). They are the uAST described in
    the Compiler Revamp slides; however in the source code I just call
    them AST because I think that is what somebody new to the project
    would expect in the long term.
  4. There is a BaseAST and tag etc just like in the current compiler. The
    isSomeAST and toSomeAST operations are now methods on BaseAST. These
    are generated by the preprocessor using the file ASTClassesList.h to
    run a macro per AST type.
  5. Only BaseAST actually contains a field for child AST nodes. The
    BaseAST node has a vector of "owned" BaseAST children and provides
    numChildren and child(int i) methods to access this list. As a
    result there is no need to define an analogue to AST_CHILDREN_CALL
    in the current compiler. There are many ways to arrange for a generic
    traversal of the AST. I chose this one as a starting point because it
    is the only one that I know of that will allow us to create a simple
    generic routine to read serialized AST from a file. It additionally
    seems to match the MLIR approach (the IR is actually quite flexible
    but defining particular types on it restricts e.g. the types of
    children). I am expecting to also have AST visitors but they aren't
    defined yet.
  6. The current compiler uses relatively few AST nodes. In some ways
    that's nice because there are fewer concepts required to manipulate
    the AST. But it's not great when the code manipulating the AST has to
    do lots of pattern matching to find a common pattern. I think that
    over time, in the current compiler, we have been adding more and more
    AST nodes. In the new effort, the approach is to define more AST
    classes for the basic ideas reflected in the syntax. Then, these AST
    classes are arranged in an inheritance hierarchy so that the things
    that used to be easy are still easy. For example, since there is a
    parent class of all Decls includes a method giving the Symbol being
    defined, it's easy to iterate over any kind of Decl. But, code that
    needs to process only Functions can iterate over just FunctionDecls.
    The ASTClassesList.h file shows the AST types we are expecting to
    create as well as their parent classes.
  7. Similarly to the previous bullet, the current compiler represents a
    lot     of the Chapel language with primitives. I think there is still a
    place for primitives but in the new effort the most important
    primitives (like "." as an example) will have their own AST nodes.
  8. With all of these AST nodes, I'm trying to make it as easy as possible
    to define AST nodes. There are a handful of functions/methods that
    will need to be defined per AST node I'm trying to keep these to a
    minimum.
  9. These AST classes don't have public constructors. Instead, they have a
    static (type level) build method. E.g., to construct a BlockStmt,
    one would write BlockStmt::build(myBuilder, location, stmtsList).
    This strategy allows the Builder to be involved in the construction
    (e.g. assigning IDs or whatever else is needed). Currently the
    Builder argument is primarily used when constructing UniqueStrings.
    Anyway, this approach is an alternative to having a Builder class that
    has methods for all of the AST types. The idea is that this approach
    makes it easier to see how to construct one of the AST nodes in the
    documentation while still allowing the important flexibility that the
    Builder idea allows.
  10. The AST does have a class to represent a comment. This class will
    represent only comments that at a statement-list level. The idea is to
    make it easier to write tools like chpldoc.
  • [x] full local testing

Future Work:

  • PR #17609 will implement many renamings we have discussed

Reviewed by @lydia-duncan and @vasslitvinov - thanks!

Modified Files:
A compiler/next/include/chpl/AST/ASTClassesList.h

A compiler/next/include/chpl/AST/ASTTag.h
A compiler/next/include/chpl/AST/ASTTypes.h
A compiler/next/include/chpl/AST/BaseAST.h
A compiler/next/include/chpl/AST/BlockStmt.h
A compiler/next/include/chpl/AST/Builder.h
A compiler/next/include/chpl/AST/CallExpr.h
A compiler/next/include/chpl/AST/Comment.h
A compiler/next/include/chpl/AST/Decl.h
A compiler/next/include/chpl/AST/ErroneousExpr.h
A compiler/next/include/chpl/AST/Expr.h
A compiler/next/include/chpl/AST/FnCallExpr.h
A compiler/next/include/chpl/AST/ID.h
A compiler/next/include/chpl/AST/Identifier.h
A compiler/next/include/chpl/AST/Literal.h
A compiler/next/include/chpl/AST/Module.h
A compiler/next/include/chpl/AST/ModuleDecl.h
A compiler/next/include/chpl/AST/OpCallExpr.h
A compiler/next/include/chpl/AST/PrimCallExpr.h
A compiler/next/include/chpl/AST/README
A compiler/next/include/chpl/AST/Stmt.h
A compiler/next/include/chpl/AST/Symbol.h
A compiler/next/include/chpl/AST/Variable.h
A compiler/next/include/chpl/AST/VariableDecl.h
A compiler/next/lib/AST/ASTTag.cpp
A compiler/next/lib/AST/ASTTypes.cpp
A compiler/next/lib/AST/BaseAST.cpp
A compiler/next/lib/AST/BlockStmt.cpp
A compiler/next/lib/AST/Builder.cpp
A compiler/next/lib/AST/Comment.cpp
A compiler/next/lib/AST/Decl.cpp
A compiler/next/lib/AST/ErroneousExpr.cpp
A compiler/next/lib/AST/Expr.cpp
A compiler/next/lib/AST/ID.cpp
A compiler/next/lib/AST/Identifier.cpp
A compiler/next/lib/AST/Module.cpp
A compiler/next/lib/AST/ModuleDecl.cpp
A compiler/next/lib/AST/Stmt.cpp
A compiler/next/lib/AST/Symbol.cpp
A compiler/next/lib/AST/Variable.cpp
A compiler/next/lib/AST/VariableDecl.cpp

Compare: https://github.com/chapel-lang/chapel/compare/ae295b0ef565...be4f2f7ca112