search_core

# search_core

Multilingual full-text search building blocks for **plain Elixir/Ecto** — no Ash
required. Two pieces, both driven by the [`stemmers`](../stemmers) Rust NIF:

- `SearchCore.Pipeline` — turn raw text into normalized, stemmed tokens.
- `SearchCore.Tsvector` — turn those tokens into the `search_text` you index and the
  `tsquery` you search with, both from the *same* pipeline so index and query never
  drift apart.

Part of the [search_ash monorepo](../).

```elixir
SearchCore.process("Les idées mangent les chevaux", :french)
#=> ["ide", "mangent", "cheval"]

SearchCore.searchable_text("Les chevaux mangent", :french)  #=> "cheval mangent"
SearchCore.tsquery("chevaux", :french)                      #=> "cheval"
```

## Postgres wiring

Tokens are already stemmed and accent-folded, so use the `'simple'` config:

```elixir
# indexing — store this in a column, GIN-index its tsvector
searchable = SearchCore.searchable_text(title <> " " <> body, :french)

# querying
import Ecto.Query
tsquery = SearchCore.tsquery(user_input, :french)

from a in Article,
  where: fragment("to_tsvector('simple', ?) @@ to_tsquery('simple', ?)", a.search_text, ^tsquery)
```

Because both sides go through the same pipeline, a search for "chevaux" finds a row
that stored "cheval" — no "search returns nothing" surprises.

See [`examples/basic`](examples/basic) for a runnable Ecto demo (multilingual seed,
search, and an `EXPLAIN` confirming GIN index usage).