Packages

Fast Hugging Face tokenizer.json, OpenAI tiktoken, and SentencePiece model bindings for Elixir via the IREE tokenizer runtime

Current section

Files

Jump to
iree_tokenizers native iree_tokenizers_native vendor IREE_COMMIT
Raw

native/iree_tokenizers_native/vendor/IREE_COMMIT

71af3a5e41a8e265330bc693194c708cf6df4724