Branch: refs/heads/main
Revision: 774aac5
Author: arezaii
Link: dyno: add token counting and code size stats by arezaii · Pull Request #19781 · chapel-lang/chapel · GitHub
Log Message:
Merge pull request #19781 from arezaii/dyno-token-count
dyno: add token counting and code size stats
This PR adds token counting and code size stats when using the dyno parser.
The production parser has the ability to count and print tokens
for files named on the command line. There are two tests that were
failing because this functionality was not available in dyno.
To avoid relying on global state, this implementation introduces
a struct that contains all of the counting functions and variables.
The reproduction in dyno is not 100% faithful to the original
implementation w.r.t multi-line comments, mostly due to differences
in how the dyno parser handles comments. For example, dyno parses
and retains comment indicators //, /*, and */ where the old
parser discards them.
TESTING:
- [x] paratest
- [x] paratest with
--dyno(34 failures)
reviewed by @mppf - thank you!
Signed-off-by: arezaii ahmad.rezaii@hpe.com
Modified Files:
A compiler/dyno/include/chpl/parsing/parser-stats.h
A compiler/dyno/lib/parsing/parser-stats.cpp
M compiler/dyno/include/chpl/parsing/Parser.h
M compiler/dyno/include/chpl/parsing/parsing-queries.h
M compiler/dyno/lib/parsing/CMakeLists.txt
M compiler/dyno/lib/parsing/Makefile.include
M compiler/dyno/lib/parsing/Parser.cpp
M compiler/dyno/lib/parsing/ParserContext.h
M compiler/dyno/lib/parsing/flex-chpl-lib.cpp
M compiler/dyno/lib/parsing/flex-chpl-lib.h
M compiler/dyno/lib/parsing/lexer-help.h
M compiler/dyno/lib/parsing/parser-dependencies.h
M compiler/dyno/lib/parsing/parsing-queries.cpp
M compiler/parser/parser.cpp
Compare: https://github.com/chapel-lang/chapel/compare/4d20913c5d6c...774aac5d5731