starfuzz

<p align="center">
  <img src="https://raw.githubusercontent.com/gleam-foundry/starfuzz/main/assets/images/starfuzz-logo.png" width="400" alt="starfuzz logo">
</p>

# starfuzz


A reusable, well-tested, pure Gleam toolkit for fuzzy string matching, text normalization, string distance, similarity scoring, ranking, fuzzy search, phonetic comparison, and entity resolution.

Supports both **Erlang (BEAM)** and **JavaScript** compilation targets.

[![Package Version](https://img.shields.io/hexpm/v/starfuzz)](https://hex.pm/packages/starfuzz)
[![Hex Docs](https://img.shields.io/badge/hex-docs-ffaff3)](https://hexdocs.pm/starfuzz/)

`starfuzz` provides a unified, coherent API for string similarity, search, and entity resolution in Gleam, avoiding Spotify-specific or domain-specific assumptions.

## Contents

- [Start here](#start-here)
- [The default configuration presets](#the-default-configuration-presets)
- [Why Normalization matters](#why-normalization-matters)
- [Why Tokenization matters](#why-tokenization-matters)
- [Why Phonetics matter](#why-phonetics-matter)
- [Distance vs. Similarity](#distance-vs-similarity)
- [Why Resolver Thresholds & Confidence matter](#why-resolver-thresholds--confidence-matter)
- [Quick Start Examples](#quick-start-examples)
- [Module API Reference](#module-api-reference)
- [Installation](#installation)
- [Operational guidance](#operational-guidance)
- [License](#license)

## Start here

If you only need to run a basic fuzzy search over a list of strings:

```gleam
import starfuzz/search

pub fn search_names(query: String, candidates: List(String)) {
  search.strings(query, candidates)
}
```

If you need a custom search configuration with minimum score and normalizer thresholds:

```gleam
import starfuzz/search
import starfuzz/similarity
import starfuzz/normalize

pub fn search_songs(query: String, songs: List(Song)) {
  let norm = normalize.new()
    |> normalize.with_lowercase
    |> normalize.with_trim
    |> normalize.with_collapsed_whitespace

  search.new(songs, fn(s: Song) { s.title })
  |> search.using(similarity.jaro_winkler)
  |> search.with_normalizer(norm)
  |> search.with_minimum_score(0.8)
  |> search.with_limit(3)
  |> search.run(query)
}
```

If you need a multi-signal ranked score composition:

```gleam
import starfuzz/rank
import starfuzz/similarity

pub fn rank_matches(query: Song, candidates: List(Song)) {
  rank.builder()
  |> rank.add_signal("title", 0.7, fn(s: Song) { similarity.jaro(query.title, s.title) })
  |> rank.add_signal("artist", 0.3, fn(s: Song) { similarity.jaro(query.artist, s.artist) })
  |> rank.calculate(candidates)
}
```

## The default configuration presets

The submodules are structured so you do not have to assemble distance metrics, normalizers, and tokenizers manually unless you want to.

### Defaults Table

| Module | Default Configuration / Preset |
|---|---|
| `starfuzz/normalize` | `basic` preset (lowercase, trim, strip punctuation, collapse whitespace) |
| `starfuzz/tokenize` | Whitespace word splitting |
| `starfuzz/distance` | Levenshtein edit distance |
| `starfuzz/similarity` | Normalized Levenshtein similarity |
| `starfuzz/search` | Levenshtein similarity, no normalizer, limit None, min_score 0.0 |
| `starfuzz/resolver` | threshold 0.7, confidence_thresholds #(0.75, 0.9) |

---

## Why Normalization matters

Normalization transforms strings into a standard form before matching. Without normalization, simple formatting variations can completely break distance calculation.
For example, `"  Fear of the Dark!  "` and `"fear of the dark"` are semantically identical, but Levenshtein distance treats them as 6 edits apart due to leading/trailing spaces, capital letters, and the exclamation mark. Normalizing first collapses this formatting noise.

---

## Why Tokenization matters

Tokenization splits text into structural parts (characters, words, n-grams) before matching.
* **Character n-grams** (e.g. bigrams, trigrams) are highly effective for catching typographical errors or spelling variants.
* **Word n-grams** are useful for finding phrase overlaps while ignoring slight word reorderings.
`starfuzz` offers target-independent Unicode tokenization presets out of the box.

---

## Why Phonetics matter

Sometimes, words sound the same but are spelled differently (e.g., `"Robert"` and `"Rupert"`). Phonetic encoding mapping (Soundex) converts strings into structural sound representations. Comparing the Soundex codes lets you find homophone matches that edit distances might miss.

---

## Distance vs. Similarity

* **Distance Metrics** (e.g., Levenshtein, Hamming) return integers where a lower value means the strings are closer. They represent the actual number of single-character operations required to turn one string into another.
* **Similarity Metrics** return floats between `0.0` (entirely different) and `1.0` (identical). They are normalized so that they can be easily combined or compared across strings of different lengths.

---

## Why Resolver Thresholds & Confidence matter

Entity resolution maps ambiguous queries onto a known collection of entities. Setting thresholds and confidence mappings:
1. **Reduces False Positives**: Matches below the minimum threshold are automatically filtered out.
2. **Determines Confidence**: High, Medium, and Low tiers explain how close the match was to the query.
3. **Explains Scores**: Resolvers expose the contributing weights and scores of each component (e.g., title weight vs. artist weight), making matches understandable.

---

## Quick Start Examples

### Normalizing text

```gleam
import starfuzz/normalize

let norm = normalize.new()
  |> normalize.with_lowercase
  |> normalize.with_trim
  |> normalize.without_punctuation

normalize.apply(norm, "  Gleam!  ") // -> "gleam"
```

### Tokenizing text

```gleam
import starfuzz/tokenize

tokenize.words("Fear of the Dark")
// -> ["Fear", "of", "the", "Dark"]

tokenize.character_ngrams("gleam", 2)
// -> Ok(["gl", "le", "ea", "am"])
```

### Calculating similarity

```gleam
import starfuzz/similarity

similarity.jaro_winkler("dwayne", "duane")
// -> 0.84
```

### Running Entity Resolution

```gleam
import starfuzz/resolver
import starfuzz/similarity

type Song {
  Song(title: String, artist: String)
}

let song_resolver = resolver.new(fn(q: Song, c: Song) {
  [
    resolver.component("title", 0.6, similarity.jaro_winkler(q.title, c.title)),
    resolver.component("artist", 0.4, similarity.jaro(q.artist, c.artist)),
  ]
})
|> resolver.with_threshold(0.7)

let query = Song("Fear of the Dark", "Iron Maiden")
let candidates = [
  Song("Fear of the Dark (2015 Remaster)", "Iron Maiden"),
  Song("Fear of the Dark", "Graveworm"),
]

resolver.resolve(song_resolver, query, candidates)
```

---

## Module API Reference

| Module | Core Types | Main Functions |
|---|---|---|
| `starfuzz/normalize` | `Normalizer` | `new`, `with_lowercase`, `with_trim`, `apply`, `basic` |
| `starfuzz/tokenize` | `TokenizeError` | `characters`, `words`, `character_ngrams`, `word_ngrams` |
| `starfuzz/phonetic` | - | `soundex`, `same_soundex` |
| `starfuzz/distance` | `HammingError` | `levenshtein`, `optimal_string_alignment`, `longest_common_subsequence`, `hamming` |
| `starfuzz/similarity` | - | `levenshtein`, `jaro`, `jaro_winkler`, `dice`, `jaccard`, `cosine` |
| `starfuzz/search` | `Match`, `SearchBuilder` | `new`, `using`, `with_normalizer`, `run`, `strings` |
| `starfuzz/rank` | `ComponentScore`, `Ranking`, `RankBuilder` | `builder`, `add_signal`, `combine`, `calculate` |
| `starfuzz/resolver` | `Confidence`, `Resolution`, `Resolver` | `new`, `with_threshold`, `resolve`, `component` |
| `starfuzz/benchmark` | `Case`, `Algorithm`, `BenchmarkResult` | `new`, `add_algorithm`, `add_cases`, `run` |

---

## Installation

Add `starfuzz` to your project:

```sh
gleam add starfuzz
```

---

## Operational guidance

### Target Independence
`starfuzz` is fully compatible with both BEAM and JavaScript compiler targets. Every distance and similarity algorithm computes identical values on both targets.

### Stable Sorting
All list-sorting operations are index-preserved and deterministic: when two candidate items receive the exact same score, their original index in the input list determines their relative position.

### Timing Portability
`starfuzz/benchmark` uses target-specific FFIs (`os:system_time` on Erlang/BEAM and `Date.now()` on JavaScript) to compute target-independent elapsed milliseconds.

---

## License

MIT License - Copyright (c) 2026 Antonio Ognio

Made with ❤️ from 🇵🇪. El Perú es clave 🔑.