xo-tokenizer: docs + error-handling improvement

drop exceptions for return-value error object
This commit is contained in:
Roland Conybeare 2025-06-23 23:08:12 -05:00
commit 6fbfd065a2
30 changed files with 1086 additions and 162 deletions

View file

@ -0,0 +1,36 @@
.. _implementation:
.. toctree::
:maxdepth: 2
Components
==========
Library dependency tower for *xo-tokenizer*:
.. ditaa::
+-----------------+
| xo_unit |
+-----------------+
| xo_indentlog |
+-----------------+
| xo_cmake |
+-----------------+
Install instructions :doc:`here<install>`
Abstraction tower for *xo-tokenizer* components:
.. ditaa::
:--scale: 0.85
+-----------------------------------------+----------+
| tokenizer | |
+-----------------------------------------+ |
| scan_result | |
+-----------------+-----------------------+ buffer |
| token | tokenizer_error | |
+-----------------+-----------------------+ |
| tokentype | span | |
+-----------------+-----------------------+----------+