-1
if this token is flyweight (and therefore does not store offset).TokenSequence.token()
but always returns a non-flyweight token
with the appropriate offset.TokenHierarchy
class with
its static methods that provide its instance for the given input source.LanguageHierarchy
that mainly defines
set of token ids and token categories for the new language
and its Lexer
.