Packages

Fast Hugging Face tokenizer.json, OpenAI tiktoken, and SentencePiece model bindings for Elixir via the IREE tokenizer runtime

Current section

Files

Jump to
iree_tokenizers native iree_tokenizers_native vendor IREE_COMMIT
Raw

native/iree_tokenizers_native/vendor/IREE_COMMIT

21a674843cd24adfdd5d5e7acb1e91ee644b45e4