Token package:parsers

Parsers that comprehend whitespace and identifier styles

idStyle    = haskellIdents { styleReserved = ... }
identifier = ident idStyle
reserved   = reserve idStyle

token p first applies parser p and then the whiteSpace parser, returning the value of p. Every lexical token (token) is defined using token, this way every parse starts at a point without white space. Parsers that use token are called token parsers in this document. The only point where the whiteSpace parser should be called explicitly is the start of the main parser in order to skip any leading white space. Alternatively, one might define token as first parsing whiteSpace and then parser p. By parsing whiteSpace first, the parser is able to return before parsing additional whiteSpace, improving laziness.

mainParser  = sum <$ whiteSpace <*> many (token digit) <* eof

Additional functionality that is needed to tokenize input while ignoring whitespace.