Building a Lexer Framework Part 2 - The state machine

Using the black magic we developed for the regex engine we can easily create a state machine for any set of lexer rules.