kura

# Kura

<p align="center">
  <img src="priv/kura.png" alt="Kura" width="600" />
</p>

Database layer for Erlang - Ecto-equivalent abstractions in pure Erlang. Pluggable backends: [`kura_postgres`](https://github.com/Taure/kura_postgres) (PostgreSQL via pgo), [`kura_sqlite`](https://github.com/Taure/kura_sqlite) (SQLite via esqlite).

## Features

- **Schema** - behaviour-based schema definitions with type metadata
- **Changeset** - cast external params, validate, track changes and errors
- **Query Builder** - composable, functional query construction
- **SQL Compiler** - parameterized SQL generation (no string interpolation)
- **Repo** - CRUD operations with automatic type conversion and PG error mapping
- **Associations** - `belongs_to`, `has_one`, `has_many`, `many_to_many` with preloading
- **Composite keys** - N-column primary keys (`key/0`) and composite foreign keys (`#kura_ref{}`)
- **Embedded Schemas** - `embeds_one`, `embeds_many` stored as JSONB
- **Multi** - atomic transaction pipelines
- **Migrations** - DDL operations with automatic module-based discovery
- **Enums** - atom-backed enum types stored as `VARCHAR`
- **Telemetry** - query logging with timing
- **Lifecycle Hooks** - before/after callbacks for insert, update, delete
- **Audit Trail** - automatic change tracking with actor context
- **Pagination** - offset-based and cursor-based pagination
- **Streaming** - server-side cursor streaming for large result sets
- **Multitenancy** - schema prefix and attribute-based tenant isolation
- **Optimistic Locking** - concurrent update conflict detection
- **Encryption at rest** - `{encrypted, Type}` fields, AES-256-GCM, key rotation

## Quick Start

### Define a Schema

```erlang
-module(user).
-behaviour(kura_schema).
-include_lib("kura/include/kura.hrl").

-export([table/0, fields/0]).

table() -> ~"users".

fields() ->
    [
        #kura_field{name = id, type = id, primary_key = true, nullable = false},
        #kura_field{name = name, type = string, nullable = false},
        #kura_field{name = email, type = string, nullable = false},
        #kura_field{name = age, type = integer},
        #kura_field{name = inserted_at, type = utc_datetime},
        #kura_field{name = updated_at, type = utc_datetime}
    ].
```

### Define a Repo

```erlang
-module(my_repo).
-behaviour(kura_repo).

-export([otp_app/0, start/0, all/1, get/2, insert/1, update/1, delete/1]).

otp_app() -> my_app.

start() -> kura_repo_worker:start(?MODULE).
all(Q) -> kura_repo_worker:all(?MODULE, Q).
get(Schema, Id) -> kura_repo_worker:get(?MODULE, Schema, Id).
insert(CS) -> kura_repo_worker:insert(?MODULE, CS).
update(CS) -> kura_repo_worker:update(?MODULE, CS).
delete(CS) -> kura_repo_worker:delete(?MODULE, CS).
```

Configure the database connection in `sys.config`:

```erlang
[{my_app, [
    {my_repo, #{
        database => ~"myapp",
        hostname => ~"localhost",
        port => 5432,
        username => ~"postgres",
        password => <<>>,
        pool_size => 10
    }}
]}].
```

### Changesets

```erlang
%% Cast and validate external params
CS = kura_changeset:cast(user, #{}, #{~"name" => ~"Alice", ~"email" => ~"alice@example.com"}, [name, email, age]),
CS1 = kura_changeset:validate_required(CS, [name, email]),
CS2 = kura_changeset:validate_format(CS1, email, ~"@"),
CS3 = kura_changeset:validate_length(CS2, name, [{min, 1}, {max, 100}]),

%% Insert
{ok, User} = my_repo:insert(CS3).
```

### Query Builder

```erlang
Q = kura_query:from(user),
Q1 = kura_query:where(Q, {age, '>', 18}),
Q2 = kura_query:where(Q1, {'or', [{role, ~"admin"}, {role, ~"moderator"}]}),
Q3 = kura_query:select(Q2, [name, email]),
Q4 = kura_query:order_by(Q3, [{name, asc}]),
Q5 = kura_query:limit(Q4, 10),

{ok, Users} = my_repo:all(Q5).
```

Supported conditions: `=`, `!=`, `<`, `>`, `<=`, `>=`, `like`, `ilike`, `matches`, `in`, `not_in`, `is_nil`, `is_not_nil`, `between`, `{'and', [...]}`, `{'or', [...]}`, `{'not', ...}`, `{fragment, SQL, Params}`.

#### Full-text search

`{Field, matches, Query}` is a full-text search: on PostgreSQL it compiles
to `to_tsvector(Field) @@ plainto_tsquery($n)`, with the query text bound
as a parameter (`plainto_tsquery` treats it as plain words, so no query
syntax is needed or trusted). Requires a backend declaring the
`full_text_search` capability.

```erlang
Q = kura_query:where(kura_query:from(article), {body, matches, ~"erlang database"}),
{ok, Hits} = my_repo:all(Q).
```

Matching uses the server's `default_text_search_config` for stemming and
stop-words. For production, add a GIN expression index so the search
doesn't recompute a `tsvector` per row:

```sql
CREATE INDEX articles_body_fts ON articles USING gin (to_tsvector('english', body));
```

#### Window functions

Use `over/2` inside `select_expr/2` for `OVER (PARTITION BY ... ORDER BY ...)`
expressions. The window function is an aggregate (`{count, '*'}`, `{sum, Field}`,
`{avg, Field}`, `{min, Field}`, `{max, Field}`) or a ranking function
(`row_number`, `rank`, `dense_rank`).

```erlang
Q = kura_query:select_expr(kura_query:from(sales), [
    {category, category},
    {row_num, kura_query:over(row_number, #{partition_by => [category], order_by => [{amount, desc}]})},
    {running_total, kura_query:over({sum, amount}, #{partition_by => [category], order_by => [{day, asc}]})}
]),

{ok, Rows} = my_repo:all(Q).
```

Requires a backend declaring the `window_functions` capability — PostgreSQL today (SQLite window rendering is planned).

### Migrations

```erlang
-module(m20240115120000_create_users).
-behaviour(kura_migration).
-include_lib("kura/include/kura.hrl").

-export([up/0, down/0]).

up() ->
    [{create_table, ~"users", [
        #kura_column{name = id, type = id, primary_key = true, nullable = false},
        #kura_column{name = name, type = string, nullable = false},
        #kura_column{name = email, type = string, nullable = false},
        #kura_column{name = age, type = integer},
        #kura_column{name = inserted_at, type = utc_datetime},
        #kura_column{name = updated_at, type = utc_datetime}
    ]},
    {create_index, ~"users", [email], #{unique => true}}].

down() ->
    [{drop_index, ~"users_email_index"},
     {drop_table, ~"users"}].
```

Run migrations:

```erlang
kura_migrator:migrate(my_repo).
kura_migrator:rollback(my_repo).
kura_migrator:status(my_repo).
```

## Type Mapping

| Kura | PostgreSQL | SQLite | Erlang |
|---|---|---|---|
| `id` | `BIGSERIAL` | `INTEGER PRIMARY KEY` | `integer()` |
| `integer` | `INTEGER` | `INTEGER` | `integer()` |
| `float` | `DOUBLE PRECISION` | `REAL` | `float()` |
| `string` | `VARCHAR(255)` | `TEXT` | `binary()` |
| `text` | `TEXT` | `TEXT` | `binary()` |
| `boolean` | `BOOLEAN` | `INTEGER` (0/1) | `boolean()` |
| `date` | `DATE` | `TEXT` (ISO 8601) | `{Y, M, D}` |
| `utc_datetime` | `TIMESTAMPTZ` | `TEXT` (ISO 8601) | `calendar:datetime()` |
| `uuid` | `UUID` | `TEXT` | `binary()` |
| `jsonb` | `JSONB` | `TEXT` | `map()` |
| `{array, T}` | `T[]` | unsupported | `list()` |
| `{encrypted, T}` | `BYTEA` | `BLOB` | `T` (encrypted at rest) |
| `{vector, N}` | `VECTOR(N)` | unsupported | `[float()]` |

SQLite values round-trip transparently via `kura_types:cast/2` (booleans 0/1 → `true`/`false`, ISO 8601 → datetime tuples, JSON text → maps).

`{vector, N}` (and dimensionless `vector`) map to [pgvector](https://github.com/pgvector/pgvector)
columns for embeddings; requires `CREATE EXTENSION vector` and a backend
declaring the `vector` capability. A value is a list of numbers, dimension-checked
on cast. Distance-query operators (`<->` / `<=>`) are not yet a first-class query
form - use a `{fragment, SQL, Params}` for nearest-neighbour ordering meanwhile.
Elements are stored as `float4`, so a value read back from a column is the
single-precision approximation of what was written, not the exact double.

## Encryption at rest

Wrap a field type in `{encrypted, Type}` to store it AES-256-GCM-encrypted
in a `BYTEA` column. Values are encrypted on write and decrypted on read;
casting and validation still run on the plaintext.

```erlang
fields() ->
    [
        #kura_field{name = id, type = id, primary_key = true},
        #kura_field{name = email, type = string},
        #kura_field{name = ssn, type = {encrypted, string}, nullable = false}
    ].
```

Supported inner types: `string`, `text`, `binary`, `uuid`, `jsonb`,
`integer`, `bigint`, `smallint`, `boolean`. Configure keys under the
`kura` app env - each a base64-encoded 32-byte key, with `active`
selecting the one used for new writes:

```erlang
[{kura, [
    {encryption, #{
        active => 1,
        keys => [{1, <<"base64-encoded-32-byte-key">>}]
    }}
]}].
```

**Rotation:** add a new key and move `active` to it. Each ciphertext
carries its key id, so existing rows keep decrypting under their original
key - no backfill. A custom keyring (KMS/Vault) can replace the default by
implementing the `kura_keyring` behaviour and setting `{kura, [{keyring, my_keyring}]}`.

**Constraints:** encrypted columns are **not queryable** and cannot carry a
unique constraint (each write has a random nonce, so equal plaintexts
produce different ciphertexts). Primary and foreign keys cannot be
encrypted. `NULL` stays `NULL` (unencrypted), so presence still leaks.
Schema introspection (`rebar3 kura gen_schemas`) cannot detect that a
`BYTEA` column was encrypted - annotate it by hand. The audit trail
redacts encrypted fields to `[encrypted]` (it keeps the "changed" signal
but never logs the value). Ciphertext is not yet bound to its row/column,
so a database-level actor could copy a ciphertext between rows of the same
column; per-column binding is planned.

## Configuration

Configure repos under the `kura` app env. Each repo is a map keyed by
its module name; pick a backend package and Kura starts the configured
pool at app boot, populating `dialect`, `pool_module`, and `driver_module`
from the aggregator automatically.

```erlang
%% sys.config — single Postgres repo
[{kura, [
    {repos, #{
        my_repo => #{
            backend => kura_backend_postgres,
            host => "localhost",
            port => 5432,
            database => "my_app_dev",
            user => "postgres",
            password => "postgres",
            pool_size => 10
        }
    }}
]}].
```

```erlang
%% sys.config — single SQLite repo
[{kura, [
    {repos, #{
        my_repo => #{
            backend => kura_backend_sqlite,
            database => <<"my_app.db">>,   %% or <<":memory:">>
            pool_size => 4
        }
    }}
]}].
```

```erlang
%% sys.config — Postgres primary + SQLite analytics
[{kura, [
    {repos, #{
        my_repo => #{
            backend => kura_backend_postgres,
            host => "localhost",
            database => "main",
            user => "postgres",
            pool_size => 10
        },
        analytics_repo => #{
            backend => kura_backend_sqlite,
            database => <<":memory:">>
        }
    }}
]}].
```

Each repo module declares itself in code:

```erlang
-module(my_repo).
-behaviour(kura_repo).
-export([otp_app/0]).
otp_app() -> my_app.
```

Queries through `my_repo` emit Postgres SQL; queries through
`analytics_repo` emit SQLite SQL. The query cache is keyed per repo so
the dialects never share entries.

UUID primary keys are auto-generated on insert when no value is
provided. The default is **UUIDv4** (random) - secure by default, since
a v4 key discloses nothing about the row. Set `uuid_version => v7` in a
repo's config to use time-ordered UUIDv7 for better index locality; note
that a v7 key embeds its creation timestamp, so avoid it for keys exposed
in URLs or public APIs. A schema's own `generate_id/0` callback still
takes precedence over both.

### Read replicas

A replica is just another repo pointed at a read replica. Mark it
`read_only => true` so writes are refused, and list it under the
primary's `replicas` so the primary can hand it out:

```erlang
{repos, #{
    my_repo => #{backend => kura_backend_postgres, host => "primary", replicas => [my_repo_ro]},
    my_repo_ro => #{backend => kura_backend_postgres, host => "replica", read_only => true}
}}
```

`read_only => true` makes every write (insert/update/delete and the
bulk and soft-delete variants) return `{error, read_only}`. Routing is
explicit - pick a replica per read and own read-after-write yourself:

```erlang
Replica = kura_repo:replica(my_repo),   %% a replica repo, or my_repo if none
{ok, Rows} = Replica:all(Q).
```

There is no automatic write/read split - reads go to whichever repo you
call, so a read that must see a just-written row should use the primary.
The guard covers the changeset write API; raw SQL via `query/3` is not
guarded, so a raw write on a replica is rejected by PostgreSQL itself.

<details>
<summary>Legacy v1.x config forms (still supported)</summary>

The flat single-repo form:

```erlang
[{kura, [
    {repo, my_repo},
    {backend, kura_backend_postgres},
    {host, "localhost"},
    {port, 5432},
    {database, "my_app_dev"},
    {user, "postgres"},
    {password, "postgres"},
    {pool_size, 10}
]}].
```

The per-app form:

```erlang
[{my_app, [
    {my_repo, #{
        backend => kura_backend_postgres,
        database => ~"my_app_dev",
        hostname => ~"localhost",
        port => 5432,
        username => ~"postgres",
        password => ~"postgres",
        pool_size => 10
    }}
]}].
```

The per-app form requires the consuming app to call `my_repo:start()`
manually. The `{repos, #{...}}` form (above) is preferred for new
projects - single-repo today, no rewrite when you add a second backend.
</details>

Migrations are discovered automatically from compiled modules implementing the `kura_migration` behaviour.

Optional telemetry/logging config:

```erlang
[{kura, [
    {log, true}  %% true | {M, F} | false (default)
]}].
```

## Plugins

- [rebar3_kura](https://github.com/Taure/rebar3_kura) - Rebar3 plugin that auto-generates migration files from schema changes. Add a field to your schema, run `rebar3 compile`, and the migration is created for you.
- [opentelemetry_kura](https://github.com/Taure/opentelemetry_kura) - OpenTelemetry instrumentation. Subscribes to Kura's telemetry events and creates spans for every database query.

## Examples

- [pet_store](https://github.com/Taure/pet_store) - A sample REST API built with Kura and Nova demonstrating schemas, changesets, queries, migrations, and associations in practice.

## Requirements

- Erlang/OTP 28+
- One backend: PostgreSQL 14+ (via [kura_postgres](https://github.com/Taure/kura_postgres)) or SQLite 3.35+ (via [kura_sqlite](https://github.com/Taure/kura_sqlite))