lang_extract

# LangExtract

[![Hex.pm](https://img.shields.io/hexpm/v/lang_extract)](https://hex.pm/packages/lang_extract)
[![Documentation](https://img.shields.io/badge/docs-hexdocs-blue)](https://hexdocs.pm/lang_extract)
[![CI](https://github.com/mdepolli/lang_extract/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/mdepolli/lang_extract/actions/workflows/ci.yml)

Extract structured data from text using LLMs, with every extraction grounded to
exact byte positions in the source. An Elixir port of
[google/langextract](https://github.com/google/langextract).

You give it two things: a **client** (which LLM to call) and a **template**
(the task definition). There is no schema DSL and no fine-tuned model — the
template is a plain-language description of what to extract, plus at least one
worked example: a sample text with the extractions you'd expect back from it.
That example is what teaches the model your class names, your span
granularity, and the output format:

```elixir
client = LangExtract.new(:claude, api_key: System.get_env("ANTHROPIC_API_KEY"))

template =
  LangExtract.template!("Extract literary works, people, and locations from the text.",
    examples: [
      %{text: "Dickens wrote Oliver Twist while living in London.",
        extractions: [
          %{class: "person", text: "Dickens"},
          %{class: "work", text: "Oliver Twist", attributes: %{"type" => "novel"}},
          %{class: "location", text: "London"}
        ]}
    ]
  )

%LangExtract.Result{spans: spans} = LangExtract.run(client, "Romeo and Juliet was written by William Shakespeare.", template)

for span <- spans do
  IO.puts("#{span.class}: \"#{span.text}\" [bytes #{span.byte_start}..#{span.byte_end}] (#{span.status})")
end
# work: "Romeo and Juliet" [bytes 0..16] (exact)
# person: "William Shakespeare" [bytes 32..51] (exact)
```

Every extraction maps back to its exact position in the source binary via
`binary_part(source, span.byte_start, span.byte_end - span.byte_start)`.

## Installation

Add `lang_extract` to your list of dependencies in `mix.exs`:

```elixir
def deps do
  [
    {:lang_extract, "~> 0.10.0"}
  ]
end
```

LangExtract uses [Req](https://hex.pm/packages/req) for HTTP calls. No
additional adapter configuration is needed.

## Quick Start

### 1. Create a client

```elixir
client = LangExtract.new(:claude, api_key: "sk-ant-...")
```

Supported providers: `:claude`, `:openai`, `:gemini`.

Provider-specific options are passed as keyword arguments:

```elixir
# OpenAI with a specific model
client = LangExtract.new(:openai, api_key: "sk-...", model: "gpt-4o")

# Gemini
client = LangExtract.new(:gemini, api_key: "gm-...")

# OpenAI-compatible endpoint (Ollama, vLLM, etc.)
client = LangExtract.new(:openai,
  api_key: "not-needed",
  base_url: "http://localhost:11434",
  json_mode: false
)
```

### 2. Define a prompt template

The template is the task definition — the only place the model learns what
"right" looks like. It has two parts:

- **`description`** — a plain-language instruction: what to extract.
- **`examples`** — worked examples: a sample `text` paired with the
  extractions you would expect from it.

The extractions inside each example are the answer key for its sample text.
They pin down everything the description leaves open: the class vocabulary
(`"condition"`, not `"diagnosis"`), the span granularity (`"diabetes"`, not
`"diagnosed with diabetes"`), which attributes to attach, and the exact
output shape — the class name becomes the JSON key in the model's reply, so
the examples also teach the wire format. A description alone would leave the
model to invent all of that.

Each extraction's `text` must appear verbatim in its example's `text`: the
examples double as alignment ground truth, and `LangExtract.template!/2`
checks this at construction, unconditionally — a template that builds is a
template whose examples align. (Use `LangExtract.template/2` for tagged
tuples instead of raises when the task definition arrives at runtime.)

```elixir
template =
  LangExtract.template!("Extract medical conditions and medications from clinical text.",
    examples: [
      %{text: "Patient was diagnosed with diabetes and prescribed metformin.",
        extractions: [
          %{class: "condition", text: "diabetes", attributes: %{"chronicity" => "chronic"}},
          %{class: "medication", text: "metformin"}
        ]}
    ]
  )
```

Examples accept string keys too, so a template loaded from JSON passes
through verbatim. The underlying structs (`LangExtract.Template`,
`Template.Example`) are public for pattern matching and introspection.

### 3. Run extraction

```elixir
source = "The patient presents with hypertension and is taking lisinopril daily."

%LangExtract.Result{spans: spans, errors: errors} = LangExtract.run(client, source, template)
```

`run/4` cannot fail — `Result.errors` is the failure channel. Every failure
stays per-chunk: chunks that fail to parse or time out land in `errors`
with their byte ranges, and the surviving chunks' spans are still
returned. Check `errors` to detect failures.

Each span contains:

| Field        | Description                                          |
| ------------ | ---------------------------------------------------- |
| `text`       | The extracted text as returned by the LLM            |
| `class`      | Entity type (e.g., `"condition"`, `"medication"`)    |
| `attributes` | Metadata the LLM attached — not byte-grounded; untrusted model output |
| `byte_start` | Inclusive byte offset in source (`nil` if not found) |
| `byte_end`   | Exclusive byte offset in source (`nil` if not found) |
| `status`     | `:exact`, `:lesser`, `:fuzzy`, or `:not_found`       |

Verify byte offsets round-trip:

```elixir
for span <- spans, span.byte_start != nil do
  extracted = binary_part(source, span.byte_start, span.byte_end - span.byte_start)
  IO.puts("#{span.class}: #{extracted}")
end
```

## Streaming

`stream/4` yields each chunk's outcome the moment it completes — first
results appear while the rest of the document is still being extracted:

```elixir
client
|> LangExtract.stream(document, template)
|> Enum.each(fn
  {:ok, chunk_result} -> send(live_view, {:spans, chunk_result.spans})
  {:error, chunk_error} -> send(live_view, {:chunk_failed, chunk_error})
end)
```

Events arrive in **completion order**, not document order; every event
carries its chunk's byte range, so consumers who need order sort and
consumers who need latency don't wait. The stream is lazy — nothing runs
until consumed, and a slow consumer naturally limits in-flight requests.

```mermaid
sequenceDiagram
    participant App as Your app
    participant LE as LangExtract
    participant C0 as Chunk 0
    participant C1 as Chunk 1
    participant C2 as Chunk 2

    App->>LE: stream(client, document, template)
    Note over LE: sentence-aware split
    LE->>C0: extract (bytes 0..N)
    LE->>C1: extract (bytes N..M)
    LE->>C2: extract (bytes M..end)
    C1-->>LE: finishes first
    LE-->>App: ok ChunkResult (byte_start N)
    C0-->>LE: LLM reply fails to parse
    LE-->>App: error ChunkError (bytes 0..N)
    C2-->>LE: done
    LE-->>App: ok ChunkResult (byte_start M)
    Note over App: completion order is not document order — sort on byte_start if you need order
```

Failure semantics differ from `run/4` deliberately: in stream mode every
failure stays per-chunk. A timed-out chunk arrives as
`{:error, %ChunkError{reason: {:task_exit, :timeout}}}` with its byte range
and the surviving chunks keep flowing — `run/4` is exactly this stream,
collected and restored to document order.

## Chunking

For documents that exceed LLM token limits, pass `:max_chunk_chars` to split the
source into sentence-aware chunks and process them in parallel:

```elixir
%LangExtract.Result{spans: spans, errors: errors} = LangExtract.run(client, long_document, template,
  max_chunk_chars: 4000,
  max_concurrency: 5
)
```

Chunk size is measured in characters, not bytes, because upstream
langextract's `max_char_buffer` counts characters: counting the same way
keeps chunk boundaries identical across the two libraries, which the
cross-library benchmarks depend on. Span offsets are always bytes, and
chunking is sentence-aware, so boundaries can't be computed from the
budget in either unit anyway. Byte offsets in the returned spans are
adjusted to reference the original source, not individual chunks.

```mermaid
sequenceDiagram
    participant App as Your app
    participant LE as LangExtract
    participant LLM as Provider

    App->>LE: run with max_chunk_chars
    Note over LE: split into sentence-aware chunks
    par Chunk 0
        LE->>LLM: prompt + chunk 0
        LLM-->>LE: JSON extractions
        Note over LE: parse, align, rebase offsets
    and Chunk 1
        LE->>LLM: prompt + chunk 1
        LLM-->>LE: JSON extractions
        Note over LE: parse, align, rebase offsets
    and Chunk 2 fails
        LE->>LLM: prompt + chunk 2
        LLM-->>LE: malformed or HTTP error
        Note over LE: ChunkError with byte range
    end
    LE-->>App: ok Result with spans and errors
    Note over App: successful spans still returned — check errors for partial failure
```

## Prompt Validation

`LangExtract.template!/2` validates at construction, so most code never calls
the validator directly. It stays public for templates assembled as structs
by hand and for re-checking under custom aligner options:

```elixir
# Returns :ok or {:error, [issues]}
:ok = LangExtract.Prompt.Validator.validate(template)

# Or raise on failure
:ok = LangExtract.Prompt.Validator.validate!(template)
```

Validation uses the production aligner, so a passing template predicts how
the same extractions align at runtime.

## Alignment Without an LLM

If you already have extraction strings (e.g., from a different source), you can
align them against source text directly:

```elixir
spans = LangExtract.align("the quick brown fox", ["quick brown", "fox"])
# [%Span{text: "quick brown", byte_start: 4, byte_end: 15, status: :exact},
#  %Span{text: "fox", byte_start: 16, byte_end: 19, status: :exact}]
```

Or parse raw LLM output and align in one step:

```elixir
raw = ~s({"extractions": [{"class": "animal", "text": "fox"}]})
{:ok, spans} = LangExtract.extract("the quick brown fox", raw)
```

Both canonical format (`class`/`text`/`attributes` keys) and dynamic-key format
(`"animal": "fox"`) are accepted. Markdown fences and `<think>` tags are
stripped automatically.

## Serialization

Store a full run faithfully — spans, errors, and usage together:

```elixir
result = LangExtract.run(client, source, template)

map = LangExtract.Serializer.result_to_map(source, result)
# %{"text" => "...", "extractions" => [...], "errors" => [...], "usage" => %{...}}

{:ok, {source, result}} = LangExtract.Serializer.result_from_map(map)
```

Known error reasons (the provider error union, pipeline errors, task
exits) serialize as tagged maps and load with their outer shape intact,
so the same patterns match live and loaded errors — payloads flatten to
strings where the original term wasn't one (e.g. `{:task_exit, :timeout}`
loads as `{:task_exit, "timeout"}`). Reasons outside the known vocabulary
fall back to their `inspect/1` rendering and load as that bare string.

For bare span lists (e.g. from `align/3`), the span-level pair applies:

```elixir
map = LangExtract.Serializer.to_map(source, spans)
# %{"text" => "...", "extractions" => [%{"class" => "...", "status" => "exact", ...}]}

{:ok, {source, spans}} = LangExtract.Serializer.from_map(map)
```

Save and load multiple span-level results as JSONL:

```elixir
LangExtract.Serializer.save_jsonl([{source1, spans1}, {source2, spans2}], "results.jsonl")
{:ok, results} = LangExtract.Serializer.load_jsonl("results.jsonl")
```

## Provider Options

All providers accept these common options:

| Option         | Default          | Description                                                              |
| -------------- | ---------------- | ------------------------------------------------------------------------ |
| `:api_key`     | From env var     | API key (falls back to provider-specific env var)                        |
| `:model`       | Provider default | Model ID                                                                 |
| `:max_tokens`  | `4096`           | Maximum response tokens                                                  |
| `:temperature` | `0` (OpenAI/Gemini); unset (Claude) | Sampling temperature. The Claude provider omits it unless set — `claude-sonnet-5` rejects non-default values |
| `:base_url`    | Provider default | API base URL                                                             |
| `:req_options` | `[]`             | Extra [Req](https://hex.pm/packages/req) options merged into the request |

Environment variable fallbacks: `ANTHROPIC_API_KEY`, `OPENAI_API_KEY`,
`GEMINI_API_KEY`.

HTTP defaults suit LLM latency: 120s receive timeout and transient retries
(429/5xx/transport errors). Override either via `:req_options`, e.g.
`req_options: [receive_timeout: 30_000, retry: false]`.

Provider-specific options:

| Provider  | Option       | Default | Description                                                                   |
| --------- | ------------ | ------- | ----------------------------------------------------------------------------- |
| `:openai` | `:json_mode` | `true`  | Enable JSON mode. Set `false` for compatible endpoints that don't support it. |

## Production: the supervised Runner

For applications extracting continuously, `LangExtract.Runner` puts a
shared request budget in your supervision tree:

```elixir
# application.ex
{LangExtract.Runner,
 name: MyApp.Extractor,
 client: LangExtract.new(:claude, api_key: key),
 max_in_flight: 20,
 rpm: 2_000}

# anywhere in the app — same shapes as run/4 and stream/4:
Runner.run(MyApp.Extractor, source, template)
Runner.stream(MyApp.Extractor, source, template)
Runner.stream_corpus(MyApp.Extractor, [{id, source}, ...], template)
```

Concurrent callers share the budget; one 429 pauses all admission until
the server's `retry-after` deadline; retries follow the runner's policy
(429 waits are free, 5xx/transport consume a per-chunk budget); stream
delivery is bounded so slow consumers throttle admission; and shutdown
drains gracefully — in-flight requests finish, unstarted chunks come back
as `%ChunkError{reason: :drained}`.

`Runner.run/4` shares `LangExtract.run/4`'s contract exactly — both return
a bare `%Result{}` and funnel every failure into per-chunk errors — the
runner just adds retries and the shared budget on the way there. See the
[production guide](guides/production.md) for sizing and the full
failure-semantics table.

## Telemetry

LangExtract emits `:telemetry` spans at document, chunk, and request level —
request events carry input/output token counts for cost tracking. See the
[Telemetry guide](guides/telemetry.md) for the event reference and examples.

## How It Works

Each chunk runs through five stages; multi-chunk documents run them in
parallel and merge the results:

```mermaid
flowchart LR
    A[Prompt Builder] --> B[LLM Provider]
    B --> C[Wire Format]
    C --> D[Parser]
    D --> E[Aligner]
```

| Stage | Role |
| ----- | ---- |
| Prompt Builder | Renders a few-shot Q&A prompt with dynamic-key examples |
| LLM Provider | Calls Claude / OpenAI / Gemini via Req |
| Wire Format | Strips fences / `<think>` tags; normalizes dynamic keys to canonical form |
| Parser | Validates and constructs `Extraction` structs |
| Aligner | Maps extraction text to byte positions (exact scan, then fuzzy fallbacks) |

The aligner mirrors upstream langextract v1.6.0 (+ #485) semantics in four
phases (see the [alignment guide](guides/alignment.md) for the full flow):

- **Occurrence DP**: Over the whole extraction list, selects one exact
  occurrence per extraction — order-preserving, non-overlapping, maximizing
  matched tokens — so repeated mentions ground to successive occurrences.
  Unplaced extractions fall through to the phases below.
- **Exact**: Linear scan for the extraction's downcased word tokens as a
  contiguous run in the source tokens. First occurrence wins.
- **Lesser (prefix match)**: When the model stitches or truncates a span, the
  longest matching token block anchored at the extraction's first token
  grounds it to its opening fragment in the source (status `:lesser`;
  disable with `accept_lesser: false`).
- **LCS fuzzy**: Longest-common-subsequence dynamic program over normalized
  (stemmed, downcased) tokens. Accepts the tightest source window with
  coverage ≥ `:fuzzy_threshold` (default 0.75) and token density ≥
  `:min_density` (default 1/3); status `:fuzzy`.

## Architecture

```
lib/lang_extract/
├── alignment/              # Tokenizer, Token, Aligner
├── pipeline/               # Parser
├── prompt/                 # Builder, Validator
├── provider/               # Claude, OpenAI, Gemini implementations
├── runner/                 # Limiter, Request, Delivery
├── chunk_error.ex          # Failed chunk: byte range + reason
├── chunk_result.ex         # Successful chunk: byte range + spans + usage
├── chunker.ex              # Sentence-aware text splitting
├── client.ex               # Configured LLM client struct
├── extraction.ex           # Core extraction struct
├── orchestrator.ex         # Pipeline wiring + chunking
├── pipeline.ex             # Extraction pipeline API
├── result.ex               # run/4 success value: spans + errors + usage
├── runner.ex               # Supervised runner with a shared request budget
├── serializer.ex           # Serialization + JSONL
├── span.ex                 # Grounded extraction: byte offsets + status
├── template.ex             # Task definition (+ Template.Example)
└── wire_format.ex          # LLM wire format (encode + decode)
```

## Stability

The docs group modules by tier; SemVer applies to the **Core API** tier.

**Core API** — the contract. `LangExtract` and `LangExtract.Runner` are the
entry points. The structs they hand out are stable to match on: `Result`,
`Span`, `ChunkError`, `ChunkResult`, and `Provider.Response` freely;
`Template`, `Template.Example`, and `Extraction` are public for matching
and introspection but constructed via `template!/2`, not struct literals.
`Client` is opaque — build it with `new/2`, hold it, pass it. The
`Provider` behaviour (callbacks plus `t:LangExtract.Provider.error/0`),
`Serializer`, `Prompt.Validator`, and the telemetry events documented in
the telemetry guide complete the contract.

**Advanced** — public and documented, best-effort stability: `WireFormat`,
`Chunker`, `Aligner`, `Pipeline`, `Prompt.Builder`. Changes land in minor
releases with changelog notice.

**Providers** — the built-in implementations behind `new/2`'s `:claude`,
`:openai`, and `:gemini`. Use them via the atom; the modules themselves
carry no stability guarantee beyond the `Provider` behaviour they
implement.

**Internal** — no guarantees: `Orchestrator`, `Pipeline.Parser`,
`Runner.{Limiter,Request,Delivery}`, `Tokenizer`, `Token`. Their docs
stay published because they explain how the library works, not because
they're API.

## Compared to the Python Original

This is an Elixir port of [google/langextract](https://github.com/google/langextract).
Key differences:

|                    | Python                            | Elixir                               |
| ------------------ | --------------------------------- | ------------------------------------ |
| Codebase           | ~4,000 LOC                        | ~2,000 LOC                           |
| Providers          | Gemini, OpenAI, Ollama            | Claude, OpenAI, Gemini               |
| Offsets            | Character positions               | Byte positions                       |
| Parallelism        | ThreadPoolExecutor                | Task.async_stream                    |
| Chunking           | Always-on (1000 chars)            | Always-on (1000 chars, configurable) |
| Alignment statuses | exact, lesser, greater (unused), fuzzy; `None` when unaligned | exact, lesser, fuzzy, not_found |
| Prompt validation  | Built-in severity levels          | Caller decides                       |

Not ported: visualization (HTML output), multi-pass extraction,
batch Vertex AI, plugin system. See the
[roadmap](https://github.com/mdepolli/lang_extract/blob/main/ROADMAP.md)
for planned improvements.

## License

MIT — see the
[LICENSE](https://github.com/mdepolli/lang_extract/blob/main/LICENSE) file
for details.