Packages

Fast Hugging Face tokenizer.json, OpenAI tiktoken, and SentencePiece model bindings for Elixir via the IREE tokenizer runtime

Current section

Files

Jump to
iree_tokenizers native iree_tokenizers_native vendor IREE_COMMIT
Raw

native/iree_tokenizers_native/vendor/IREE_COMMIT

af030e43d8343263a6c869eae32f958f229ff7af