class compiler::Tokenize
sys::Obj compiler::CompilerSupport compiler::CompilerStep compiler::Tokenize
Tokenize is responsible for parsing all the source files into a a list of tokens. Each source file is mapped to a CompilationUnit and stored in the PodDef.units field:
Compiler.srcFiles -> Compiler.pod.units
During the standard pipeline this step is called by the InitInput step.
- input
-
CompilerInput input
- make
-
new make(Compiler compiler)
- run
-
virtual override Void run()
- tokenize
-
CompilationUnit tokenize(Loc loc, Str src)