xo-tokenizer documentation
xo-tokenizer provides a tokenizer for the Schematika language.
Syntax is generally C-like, but with some important differences.
Notably, characters used for arithmetic operators (+, -, *, /)
may appear in variable names: one-of-those-days is an ordinary symbol.
Typically applications would use xo-reader to construct Schematika expressions
instead of interacting directly with xo::scm::tokenizer.
xo-tokenizer contents