sting

# Sting

SentenceTransformers inference in Elixir, built on Bumblebee, Axon, and Nx.

Load supported SentenceTransformers checkpoints, encode text, and search or mine
the resulting embeddings. Python comparison tests target **SentenceTransformers
6.1.0** for the supported inference paths. Sting implements a subset of that
library; see the [parity guide](guides/sentence-transformers-parity.md) for coverage
and differences.

## Installation

Requires Elixir 1.17 or later on a compatible Erlang/OTP runtime. The package
checks cover Elixir 1.17 / OTP 26 and Elixir 1.19 / OTP 28. The initial release
uses Bumblebee 0.8, Axon 0.9, Nx 1.0, and Tokenizers 0.5. Python parity validation
uses Elixir 1.19 / OTP 28 with Nx.BinaryBackend.

Add Sting to your application's dependencies:

```elixir
defp deps do
  [
    {:sting, "~> 0.1.0"}
  ]
end
```

Python is needed only for comparison tests, not for inference. EXLA and Torchx
are development dependencies in this repository; applications that need an
accelerated backend must add and configure it themselves. Accelerator performance
and numerical parity are not established by the BinaryBackend tests.

## Encode text

```elixir
repo = {:hf, "sentence-transformers/all-MiniLM-L6-v2"}

{:ok, model} = Sting.load_model(repo)
{:ok, tokenizer} = Sting.load_tokenizer(repo)
serving = Sting.text_embedding(model, tokenizer)

%{embedding: embedding} = Nx.Serving.run(serving, "Hello world")
Nx.shape(embedding)
#=> {384}
```

Use Sting's tokenizer and serving helpers to preserve saved prompts, the first
module's tokenizer location, and sequence limits. A list of texts returns a list
of `%{embedding: tensor}` maps. Local checkpoints use
`{:local, "/absolute/path/to/checkpoint"}` instead of a Hugging Face repository.

## Query and document search

The following continues with the loaded `model` and `tokenizer`:

```elixir
queries =
  model
  |> Sting.encode_queries(tokenizer, ["What runs on the BEAM?"])
  |> Enum.map(& &1.embedding)
  |> Nx.stack()

documents =
  model
  |> Sting.encode_corpus(tokenizer, ["Elixir runs on the BEAM.", "Rust compiles to native code."])
  |> Enum.map(& &1.embedding)
  |> Nx.stack()

hits = Sting.semantic_search(queries, documents, top_k: 2)
# One result list per query; each result contains :corpus_id and :score.
```

Query encoding selects a saved `"query"` prompt when present. Corpus encoding
tries `"document"`, `"passage"`, then `"corpus"`. Both otherwise use the saved
default. Explicit `:prompt` or `:prompt_name` options take precedence.

```elixir
serving = Sting.text_embedding(model, tokenizer, prompt: "Represent this query: ")
serving = Sting.text_embedding(model, tokenizer, prompt: false)
```

For a checkpoint that defines a saved `"query"` prompt, select it with
`Sting.text_embedding(model, tokenizer, prompt_name: "query")`. Named prompts
must exist in the checkpoint. Pooling with `include_prompt: false` excludes
prompt tokens.

## Sequence limits and compilation

Saved `max_seq_length` limits are applied before padding. Override the limit
explicitly with `:max_seq_length`. Compilation buckets select the smallest
sufficient padded shape; the largest bucket also caps the token limit.

```elixir
serving =
  Sting.text_embedding(model, tokenizer,
    max_seq_length: 64,
    compile: [batch_size: 2, sequence_length: [16, 32, 64]]
  )

Nx.Serving.run(serving, ["Hello world", "A second sentence"])
```

## Supported features

| Area | API and scope |
| --- | --- |
| Dense checkpoints | Transformer plus Pooling, Dense, Normalize, LayerNorm, and inference-only Dropout |
| Pooling | CLS, max, mean, mean-sqrt-length, weighted mean, and last token |
| Static and layer-weighted embeddings | StaticEmbedding / Model2Vec and WeightedLayerPooling; local fixture coverage |
| Sparse checkpoints | `Sting.Sparse`: MLMTransformer + SpladePooling, or SparseStaticEmbedding |
| Multi-vector checkpoints | `Sting.MultiVector`: token embeddings, masking, hierarchical token pooling, MaxSim / MeanMaxSim |
| Similarity | `Sting.Similarity`: cosine, dot, negative Euclidean/Manhattan distance, pairwise variants and pairwise angle |
| Search and mining | `Sting.Search` and `Sting.Mining`: top-k retrieval, paraphrase mining, communities, hard negatives |
| Quantization | `Sting.Quantization`: binary / unsigned binary packing, int8 calibration, Hamming distance |
| Embedding transforms | `truncate_dim` with renormalization and `fuse_dense: true` for consecutive bias-free linear layers |
| Cross encoding | `Sting.cross_encoding/3` delegates to Bumblebee; no SentenceTransformers parity claim |

Architecture support comes from Bumblebee. Supported modules do not imply that
every checkpoint containing them can be loaded. Training, fine-tuning,
multimodal models, ONNX/OpenVINO, and arbitrary custom Python modules are outside
the initial release.

### Sparse and multi-vector encoders

For a supported sparse checkpoint, use `Sting.Sparse.load_model/2`,
`Sting.Sparse.load_tokenizer/2`, and `Sting.Sparse.encode/4`. Sparse output is a
dense Nx vocabulary vector containing zeros. `Sting.Sparse.semantic_search/3`
uses dot-product similarity.

For a supported multi-vector checkpoint, use `Sting.MultiVector.load_model/2`
and `Sting.load_tokenizer/2`, then `Sting.MultiVector.encode_query/4` or
`Sting.MultiVector.encode_document/4`. These return a token matrix per text.
`Sting.MultiVector.maxsim/2` and `Sting.MultiVector.mean_maxsim/2` score those
representations.

Hierarchical token pooling runs on the host and must be the final module.
Multiple MultiVectorMask modules are rejected. See the
[parity guide](guides/sentence-transformers-parity.md) for tested behavior and
output shapes.

### Mining and quantization

Mining consumes embeddings, rather than Python Dataset or model objects:

```elixir
embeddings = Nx.tensor([[1.0, 0.0], [0.9, 0.1], [0.0, 1.0]])
pairs = Sting.paraphrase_mining(embeddings, top_k: 2, max_pairs: 3)
communities = Sting.community_detection(embeddings, threshold: 0.9, min_community_size: 2)
packed = Sting.quantize(embeddings, :ubinary)
```

Hard-negative mining requires one positive embedding per query and returns
indices and scores. Search and mining expose chunk sizes; reduce them for large
embeddings or memory-constrained hosts. Equal-score cutoff membership can differ
from Python or between Nx backends.

## Development and parity tests

```sh
mix deps.get
mix format --check-formatted
mix test --exclude python --exclude slow --exclude network
mix docs --warnings-as-errors
```

For Python parity:

```sh
python3 -m venv .venv
.venv/bin/pip install sentence-transformers==6.1.0 torch numpy datasets
STING_PYTHON="$PWD/.venv/bin/python" mix test --exclude network
```

The bridge also supports `uv` and checks the SentenceTransformers version before
running comparisons. Network-tagged tests additionally fetch public Hugging Face
checkpoints; run the full suite with `STING_PYTHON="$PWD/.venv/bin/python" mix test`.
See the [parity guide](guides/sentence-transformers-parity.md) for the evidence
behind each compatibility claim.

A local Bumblebee checkout is explicitly opt-in through `BUMBLEBEE_PATH`.
Release builds use the published Hex dependency; unset this variable before
building a package.

## Release notes and license

See the [changelog](CHANGELOG.md) and [release procedure](guides/releasing.md).

Licensed under [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0).
The license text is included in the package. Model weights retain their own licenses.