Packages

tokenizers

0.5.1

0.5.1 0.5.0 0.4.0 0.3.2 0.3.1 0.3.0 0.2.0 0.1.2 0.1.1 0.1.0

Bindings to Hugging Face Tokenizers for Elixir

HexDocs

Current section

Files

Jump to

Readme 10 Versions 3 Dependencies 11 Dependants Files Activity

tokenizers native ex_tokenizers .cargo config.toml

Raw

native/ex_tokenizers/.cargo/config.toml

[target.x86_64-apple-darwin]
rustflags = [
    "-C", "link-arg=-undefined",
    "-C", "link-arg=dynamic_lookup",
]

[target.aarch64-apple-darwin]
rustflags = [
    "-C", "link-arg=-undefined",
    "-C", "link-arg=dynamic_lookup",
]

[target.x86_64-unknown-linux-musl]
rustflags = [
  "-C", "target-feature=-crt-static"
]

[target.aarch64-unknown-linux-musl]
rustflags = [
  "-C", "target-feature=-crt-static"
]

Search filters

Type any of these into the search box. They can be combined with free text which searches package names and descriptions.

Operator	Description	Example
name:	Match package name. Supports * wildcards and repo/package form	name:phx* or name:hexpm/phoenix
description:	Full-text search of package descriptions	description:auth
depends:	Packages depending on a given package. Supports repo:package form	depends:ecto or depends:hexpm:ecto
build_tool:	Filter by build tool	build_tool:mix
updated_after:	Packages updated after an ISO8601 datetime	updated_after:2025-01-01T00:00:00Z
extra:	Match custom metadata (key,value). Nested keys are separated by commas	extra:license,MIT

Wrap values in double quotes to include spaces, e.g. name:"my package".