audit_trail

# AuditTrail

Structured audit logging, database change tracking, and crash reporting for Elixir/Phoenix applications. Supports multiple storage backends — logs can be shipped to [Grafana Loki](https://grafana.com/oss/loki/), stored in your existing PostgreSQL database, or stored in TimescaleDB for automatic time partitioning and compression.

---

## Why you need it

Most apps bolt on audit logging as an afterthought — scattered `Logger.info` calls, a `logs` database table that grows unbounded, or nothing at all. AuditTrail gives you:

- **Who did what, when, and from where** — every event carries actor identity, timestamp, IP, and user agent
- **Field-level database change tracking** — see exactly which fields changed, before and after, on any Ecto schema
- **Controller-level auditing without clutter** — one `plug` declaration at the top of a controller covers all its actions with no per-function code
- **Crash alerting** — OTP process crashes are intercepted, deduplicated, and emailed to the right team
- **Queryable history** — read logs back from Loki filtered by user, event type, date range, or free text
- **Zero-risk emit** — every emit is wrapped in a try/rescue; a logging failure never breaks your request

---

## Table of contents

- [Quick start](#quick-start-5-minutes-no-loki-required) — see a real log row in 5 minutes, no new infrastructure
- [Core concepts](#core-concepts) — the mental model, and "which piece do I actually need?"
- [Infrastructure requirements](#infrastructure-requirements) — only relevant if you pick the Loki adapter
- [Installation](#installation)
- [Configuration](#configuration)
- [Storage adapters](#storage-adapters) — [Loki](#loki-adapter-default), [Postgres](#postgres-adapter), [TimescaleDB](#timescaledb-adapter), [custom](#writing-a-custom-adapter)
- [Usage](#usage) — the five instrumentation levels
  - [1. Schema level](#1-schema-level--automatic-field-level-change-tracking) — automatic DB change tracking
  - [2. Controller level](#2-controller-level--declarative-plug) — declarative plug
  - [3. API / service level](#3-api--service-level--manual-emit) — manual `emit`/`monitor`
  - [4. Pipeline level](#4-pipeline-level--blanket-http-logging) — log every request
  - [5. LiveView level](#5-liveview-level--declarative-handle_event-auditing) — declarative `handle_event`
- [Reading logs back](#reading-logs-back)
- [Testing](#testing) — asserting on audit events in your test suite
- [Loki stream labels](#loki-stream-labels)
- [Crash reporting](#crash-reporting)
- [Limitations](#limitations) — what this library deliberately doesn't do
- [Troubleshooting](#troubleshooting)

---

## Quick start (5 minutes, no Loki required)

The fastest way to see a real audit log end to end: store it in the Postgres you already have. No Loki, no Grafana, no `docker-compose`.

```elixir
# 1. mix.exs
{:audit_trail, path: "../audit_trail"}   # or git: — see Installation below
```

```elixir
# 2. config/config.exs (or runtime.exs)
config :audit_trail,
  app_name:        "my_app",
  storage_adapter: {AuditTrail.Adapters.Postgres, repo: MyApp.Repo}
```

```bash
# 3. Fetch, generate the migration, migrate
mix deps.get
mix audit_trail.gen.migration
mix ecto.migrate
```

```elixir
# 4. Log something from anywhere — a controller, a LiveView, an Oban job, iex
AuditTrail.emit("user:signed_up", %{resource_id: user.id, plan: "free"})

# 5. Read it back
{:ok, logs} = AuditTrail.get_logs(%{type: "user:signed_up"})
```

That's it — a real, queryable audit log with five lines of config. Once this feels good, keep reading:

- Want every change to a DB row tracked automatically, with no code in your business logic? → [Schema level](#1-schema-level--automatic-field-level-change-tracking)
- Want controller actions logged declaratively instead of hand-writing `emit` calls everywhere? → [Controller level](#2-controller-level--declarative-plug)
- Want Grafana dashboards and LogQL instead of SQL? → switch `storage_adapter:` to Loki, see [Storage adapters](#storage-adapters)

---

## Core concepts

Three ideas cover almost everything below — read this before the deep-dive sections if you're new to the library.

- **Event** — one audit log entry. An `event_type` string (`"item:approved"`, `"schema:update"`, `"payment:processed"`), an actor, a timestamp, and a free-form payload. Every function in this library — `emit`, `monitor`, schema tracking, both plugs — just builds one of these and hands it to the buffer. You will mostly interact with events, not with any of the machinery that ships them.
- **Actor** — who did it. Set once per request or job with `AuditTrail.set_actor/1`; every event logged in that process afterward picks it up automatically, no need to pass it around. Defaults to `"anonymous"` if you never set it. (There's also `AuditTrail.set_tenant/1` for multi-tenant apps — same idea, for an org/tenant id instead of a user.)
- **Adapter** — where events end up. Loki, Postgres, TimescaleDB, or an in-memory Test adapter — swap with a single config line, and `AuditTrail.get_logs/1` works identically regardless of which one you're on.

Everything else in this README is either "how do I produce an event" (the five instrumentation levels) or "how do I configure/query where events go" (adapters, filters).

### Which instrumentation level do I need?

| You want to... | Use | Effort |
|---|---|---|
| Track every change to a DB row automatically | [Schema level](#1-schema-level--automatic-field-level-change-tracking) — `use AuditTrail.Schema` | Add one line to the schema, one to the Repo |
| Log specific controller actions ("item approved", "user logged in") | [Controller level](#2-controller-level--declarative-plug) — `plug AuditTrail.ControllerAudit` | One `plug` declaration, zero code in the action |
| Same, but for LiveView `handle_event` | [LiveView level](#5-liveview-level--declarative-handle_event-auditing) — `on_mount {AuditTrail.LiveViewAudit, ...}` | One `on_mount`, zero code in the callback |
| Log something with no schema and no controller — a background job, a webhook, a third-party API call | [API / service level](#3-api--service-level--manual-emit) — `AuditTrail.emit/2` | One explicit call at the point it happens |
| Log literally every HTTP request, unconditionally | [Pipeline level](#4-pipeline-level--blanket-http-logging) — `plug AuditTrail.Plug` | One `plug` in your router pipeline |

These aren't mutually exclusive — most real apps use two or three together (e.g. schema tracking on a couple of sensitive tables, the controller plug for actions that matter, `emit/2` for one-off business events like a payment webhook). Pick based on *where the event naturally happens in your code*, not on trying to standardize on one mechanism.

### Which storage adapter do I need?

| If you... | Use |
|---|---|
| Don't want to stand up new infrastructure | **Postgres** — logs live in your existing DB, see [Quick start](#quick-start-5-minutes-no-loki-required) |
| Expect high volume, or want old logs auto-compressed | **TimescaleDB** — same as Postgres, plus time partitioning |
| Already run Grafana/Loki, or want LogQL + dashboards | **Loki** (the default) — see [Infrastructure requirements](#infrastructure-requirements) |
| Are writing tests and want to assert what got logged | **Test** — in-memory, zero setup, see [Testing](#testing) |

Full setup for each is under [Storage adapters](#storage-adapters).

---

## Infrastructure requirements

> Skip this section entirely if you're using the Postgres, TimescaleDB, or Test adapter — it only applies to the Loki adapter (the default, but not required).

AuditTrail defaults to **Grafana Loki** as its storage backend, but this is optional. If you configure the Postgres or TimescaleDB adapter, no Loki instance is needed. See [Storage adapters](#storage-adapters) for how to switch.

| Component | Purpose |
|-----------|---------|
| **Loki** | Log storage and query engine |
| **Grafana** (optional) | Visual log explorer over Loki |
| **MinIO / S3** (optional) | Object storage backend for Loki in production |

<!--### Minimum Loki config (filesystem storage, local dev)

```yaml
auth_enabled: false

server:
  http_listen_port: 3100
  grpc_listen_port: 9096

common:
  path_prefix: /tmp/loki
  storage:
    filesystem:
      chunks_directory: /tmp/loki/chunks
      rules_directory:  /tmp/loki/rules
  replication_factor: 1
  ring:
    instance_addr: 127.0.0.1
    kvstore:
      store: inmemory

schema_config:
  configs:
    - from: 2024-01-01
      store: tsdb
      object_store: filesystem
      schema: v13
      index:
        prefix: index_
        period: 24h

limits_config:
  allow_structured_metadata: false
```-->

### With S3 / MinIO (production)

```yaml
common:
  storage:
    s3:
      endpoint: http://minio:9000
      bucketnames: loki-data          # ← "bucketnames" not "buckets" (Loki 3.x)
      access_key_id: YOUR_KEY
      secret_access_key: YOUR_SECRET
      s3forcepathstyle: true
      insecure: true                  # set false in real production
```

> **Loki 3.x gotcha:** the S3 field is `bucketnames`, not `buckets`. The wrong name causes Loki to crash-loop on startup with `field buckets not found in type aws.S3Config`.

---

## Installation

### 1. Add to `mix.exs`

```elixir
# Monorepo / local path
{:audit_trail, path: "../audit_trail"}

# Git
{:audit_trail, git: "https://github.com/your-org/audit_trail.git", tag: "v0.1.0"}
```

```bash
mix deps.get
mix deps.compile audit_trail
mix compile
```

> **Recompile tip:** when you add new source files to the library, run `mix deps.compile audit_trail --force && mix compile --force`. The `plug` macro calls `init/1` at **compile time**, so the library must be fully compiled before the host app compiles its controllers.

### 2. Do NOT add to your supervision tree

AuditTrail is an OTP application — it starts itself when listed as a mix dependency. Adding it to `application.ex` causes `{:already_started, pid}`:

```elixir
# ✗ Wrong
children = [
  {AuditTrail, []},  # remove this
  MyApp.Repo,
]

# ✓ Correct — just the mix dep, nothing in children
```

---

## Configuration

All config lives under `:audit_trail` in your `config/*.exs` files.

### Required

```elixir
config :audit_trail,
  app_name:      "my-app",
  loki_push_url: System.get_env("LOKI_PUSH_URL"),   # http://localhost:3100/loki/api/v1/push
  loki_read_url: System.get_env("LOKI_READ_URL")    # http://localhost:3100/loki/api/v1/query_range
```

### Buffer tuning (optional)

```elixir
config :audit_trail,
  enabled:         true,    # set false to disable all shipping (dev/test)
  partition_count: 8,       # GenServer buffer partitions (default 8)
  batch_size:      100,     # flush after this many entries (default 100)
  flush_interval:  2_000,   # flush every N ms even if batch not reached (default 2000)
  max_buffer:      5_000,   # drop logs when a partition's buffer exceeds this (default 5000, PER PARTITION — real cap is partition_count * max_buffer)
  max_dead_letter: 10_000   # in-memory dead-letter capacity (default 10000)
```

### Shipper retries (optional)

```elixir
config :audit_trail,
  shipper_timeout: 5_000,   # HTTP timeout per Loki push attempt in ms (default 5000)
  shipper_retries: 3        # retries with exponential backoff before dead-letter (default 3)
```

### Crash reporting (optional)

```elixir
config :audit_trail,
  crash_reporting:      true,
  dedup_window_minutes: 30,          # suppress duplicate crash emails within this window
  dedup_max_entries:    1_000,

  mailer_adapter: {AuditTrail.Adapters.Swoosh, []},
  swoosh_mailer:  MyApp.Mailer,
  from_email:     {"MyApp Alerts", "alerts@myapp.com"},

  crash_routing: [
    %{match: {:module, ~r/MyApp\.Items/},   to: ["items-team@myapp.com"]},
    %{match: {:module, ~r/MyApp\.Auth/},    to: ["security@myapp.com"]},
    %{match: :default,                      to: ["admin@myapp.com"]}
  ]
```

### Recommended per-environment setup

```elixir
# config/dev.exs
config :audit_trail,
  enabled:       true,   # or false to silence all Loki calls during local dev
  loki_push_url: "http://localhost:3100/loki/api/v1/push",
  loki_read_url: "http://localhost:3100/loki/api/v1/query_range"

# config/test.exs
config :audit_trail,
  enabled: false         # never ship logs during tests

# config/runtime.exs (production)
config :audit_trail,
  enabled:       true,
  loki_push_url: System.fetch_env!("LOKI_PUSH_URL"),
  loki_read_url: System.fetch_env!("LOKI_READ_URL")
```

When `enabled: false` the buffer still accepts log calls (no errors, no warnings) but the shipper never fires. Safe to leave logging calls in your code in all environments.

---

## Storage adapters

AuditTrail ships with four storage adapters. All four implement the same interface, so `AuditTrail.get_logs/1` works identically regardless of which backend you use.

| Adapter | Requires | Best for |
|---------|----------|----------|
| `AuditTrail.Adapters.Loki` | Grafana Loki | Production observability stacks, long-term log retention, Grafana dashboards |
| `AuditTrail.Adapters.Postgres` | Your existing Ecto Repo | Apps already on Postgres that don't want to run Loki |
| `AuditTrail.Adapters.TimescaleDB` | TimescaleDB extension | High-volume logs, automatic compression, time-range query performance |
| `AuditTrail.Adapters.Test` | Nothing | Asserting on audit events in your test suite — see [Testing](#testing) |

The default is Loki. No adapter config is needed if you set `loki_push_url`.

---

### Loki adapter (default)

```elixir
# config/runtime.exs — Loki is the default, this line is optional
config :audit_trail,
  storage_adapter: AuditTrail.Adapters.Loki,  # can omit this line
  loki_push_url:   System.fetch_env!("LOKI_PUSH_URL"),
  loki_read_url:   System.fetch_env!("LOKI_READ_URL")
```

No migration needed. Logs are stored as labelled JSON streams in Loki and are queryable with LogQL via Grafana or the built-in reader.

---

### Postgres adapter

Stores logs in an `audit_logs` table in your own PostgreSQL database. No external services required.

#### 1. Configure

```elixir
config :audit_trail,
  storage_adapter: {AuditTrail.Adapters.Postgres, repo: MyApp.Repo}
```

The `repo:` option is required — pass your app's `Ecto.Repo` module.

#### 2. Generate and run the migration

```bash
mix audit_trail.gen.migration
mix ecto.migrate
```

This creates a timestamped migration file in `priv/repo/migrations/`. The migration adds the `audit_logs` table with these columns:

| Column | Type | Description |
|--------|------|-------------|
| `id` | bigserial | Auto-incrementing primary key |
| `event_id` | text | UUID assigned at emit time |
| `event_type` | text | e.g. `"auth:login"` |
| `actor_id` | text | User ID or `"anonymous"` |
| `actor_name` | text | Display name of the actor |
| `actor_email` | text | Email of the actor |
| `payload` | jsonb | Full structured event payload |
| `inserted_at` | timestamptz | When the event was recorded |

A GIN index on `payload` is added automatically so JSON field searches are fast — this is also what makes the optional `resource`/`operation` filters (see [Reading logs back](#reading-logs-back)) work without a schema change.

#### 3. Options for the migration task

```bash
# Target a specific repo
mix audit_trail.gen.migration --repo MyApp.Repo

# Write to a custom path
mix audit_trail.gen.migration --migrations-path priv/audit_trail/migrations
```

---

### TimescaleDB adapter

TimescaleDB is PostgreSQL with automatic time-based partitioning. The adapter is functionally identical to the Postgres adapter — it differs only in the migration, which also calls `create_hypertable`.

#### 1. Configure

```elixir
config :audit_trail,
  storage_adapter: {AuditTrail.Adapters.TimescaleDB, repo: MyApp.Repo}
```

#### 2. Generate and run the migration

```bash
mix audit_trail.gen.migration
mix ecto.migrate
```

The generated migration includes the standard table plus:

```sql
SELECT create_hypertable('audit_logs', 'inserted_at', chunk_time_interval => INTERVAL '1 day')
```

#### 3. Enable compression (optional, after migrating)

Run these directly in your database to compress chunks older than 7 days:

```sql
ALTER TABLE audit_logs SET (
  timescaledb.compress,
  timescaledb.compress_segmentby = 'event_type, actor_id'
);

SELECT add_compression_policy('audit_logs', INTERVAL '7 days');
```

The `compress_segmentby` columns match the fields you filter on most — queries on those fields stay fast even against compressed chunks.

#### Why TimescaleDB over plain Postgres?

| Feature | Postgres | TimescaleDB |
|---------|----------|-------------|
| Time-range queries | Table scan (unless indexed) | Chunk pruning — only relevant chunks scanned |
| Storage for old data | Full row size always | Compressed chunks (4–20× compression typical) |
| Partition management | Manual | Automatic by time interval |

TimescaleDB is the better choice when you expect more than a few million log rows or run frequent date-range queries (the default pattern for audit logs). It is standard Postgres-compatible — your existing `mix ecto.migrate` and all Ecto queries work without modification.

---

### Switching adapters

You can switch adapters at any time by changing the config. Logs already written are not migrated — the new adapter starts fresh from the moment the config change takes effect. For a zero-downtime switch, deploy the new config and run the migration before cutting over traffic.

### Writing a custom adapter

Implement the `AuditTrail.Adapter` behaviour:

```elixir
defmodule MyApp.AuditAdapter do
  @behaviour AuditTrail.Adapter

  @impl true
  def push(logs) do
    # logs is a list of sanitized map()s with at minimum:
    #   :event_type, :actor_id, :actor_name, :actor_email, :timestamp
    :ok
  end

  @impl true
  def get_logs(filters) do
    # filters: %{actor_id:, type:, status:, search:, start_date:, end_date:, limit:}
    # Must return {:ok, [%{timestamp:, labels: map(), payload: map()}]}
    #         or {:error, reason}
    {:ok, []}
  end

  @impl true
  def child_spec(_opts), do: nil  # return nil if no supervised process needed
end
```

Then configure it:

```elixir
config :audit_trail, storage_adapter: MyApp.AuditAdapter
# or with opts:
config :audit_trail, storage_adapter: {MyApp.AuditAdapter, some_option: true}
```

The output shape of `get_logs/1` must match the structure the rest of the system expects:

```elixir
%{
  timestamp: ~U[2026-06-12 10:34:21Z],          # DateTime
  labels:  %{"app" => "...", "type" => "..."},   # string keys
  payload: %{"event_type" => "...", ...}          # string keys, full event data
}
```

---

## Usage

There are five levels at which you can instrument your app. They are independent — use as many or as few as you need.

---

### 1. Schema level — automatic field-level change tracking

The zero-effort layer. Mark a schema, and every `insert`, `update`, and `delete` through your Repo is recorded automatically with a before/after diff.

#### Instrument your Repo

```elixir
defmodule MyApp.Repo do
  use Ecto.Repo, otp_app: :my_app, adapter: Ecto.Adapters.Postgres
  use AuditTrail.RepoWatcher    # add this line
end
```

`RepoWatcher` overrides `insert/2`, `update/2`, `delete/2` (and their bang variants). Operations on non-tracked schemas pass through completely unchanged.

#### Mark schemas to track

```elixir
defmodule MyApp.Items.Item do
  use Ecto.Schema
  use AuditTrail.Schema, track: [:status, :condition, :approved_by]

  schema "items" do
    field :name,           :string
    field :status,         :string     # tracked ✓
    field :condition,      :string     # tracked ✓
    field :approved_by,    :string     # tracked ✓
    field :internal_notes, :string     # NOT tracked
  end
end
```

`track: :all` tracks every field. A list of atoms tracks only those fields — useful to exclude high-frequency or irrelevant fields like `updated_at`.

#### Embeds and associations are diffed too

`embeds_one`/`embeds_many` and `has_one`/`has_many`/`belongs_to`/`many_to_many` (via `cast_assoc`/`put_assoc`/`cast_embed`) are diffed recursively, not dumped as an opaque struct:

```elixir
defmodule MyApp.Items.Item do
  use Ecto.Schema
  use AuditTrail.Schema, track: :all

  schema "items" do
    field :status, :string
    embeds_one :shipping_address, MyApp.Items.Address
    has_many :line_items, MyApp.Items.LineItem
  end
end
```

Updating `shipping_address.city` and adding a new `line_items` row produces:

```elixir
%{
  "shipping_address" => %{
    "action"  => "update",
    "record_id" => "addr-1",
    "changes" => %{"city" => %{"from" => "Nairobi", "to" => "Mombasa"}}
  },
  "line_items" => [
    %{"action" => "insert", "changes" => %{"sku" => %{"from" => nil, "to" => "ABC-123"}}}
  ]
}
```

`action` per nested entry is `"insert"`, `"update"`, `"delete"`, or `"replace"` (Ecto's own terms for what happened to that embed/association). The parent's `track:` field list only filters which *top-level* fields are tracked — once a top-level field like `shipping_address` is in scope, all of its nested fields are included *unless the nested schema opts into its own filtering*:

```elixir
defmodule MyApp.Items.Address do
  use Ecto.Schema
  use AuditTrail.Schema, track: [:city, :country]   # filters this embed's own fields

  embedded_schema do
    field :city,           :string
    field :country,        :string
    field :contact_phone,  :string   # NOT tracked, even though shipping_address is
  end
end
```

Embeds/associations don't go through the Repo directly, so this doesn't give them independent tracking (no standalone `schema:update` event for just the embed) — it only scopes which of their fields show up in the parent's diff.

#### What gets logged

| Field | Example value |
|-------|--------------|
| `event_type` | `"schema:update"` |
| `actor_id` | `"550e8400-e29b-41d4-a716-446655440000"` |
| `schema` | `"Elixir.MyApp.Items.Item"` |
| `resource` | `nil` unless `resource:` is set (see below) |
| `resource_id` | `"abc-123"` — same value as `record_id`, kept under both names so filters can use either |
| `operation` | `"insert"` / `"update"` / `"delete"` |
| `record_id` | `"abc-123"` |
| `changes` | `{"status": {"from": "pending", "to": "approved"}}` |

The actor is whatever was last set via `AuditTrail.set_actor/1` in the current process. Defaults to `"anonymous"`.

#### Naming the resource (optional)

`schema` already identifies what was changed (the full module name), but that's noisy to filter on. Pass `resource:` for a short, business-facing label instead:

```elixir
defmodule MyApp.Payments.Payment do
  use Ecto.Schema
  use AuditTrail.Schema, track: [:status, :amount], resource: "payment"
  # ...
end
```

This is purely additive — omit `resource:` and tracking behaves exactly as before, with `resource` simply absent (`nil`) from the payload.

#### `Repo.transaction` and `Ecto.Multi` are commit-safe

`RepoWatcher` also wraps `transact/2` — the function both `Repo.transaction/2` and `Ecto.Multi` funnel through. Any audit log produced while a transaction is still open (an `Ecto.Multi.insert/update/delete` step, or a manual `AuditTrail.emit`/`monitor` call made inside the transaction function) is held, not shipped immediately:

```elixir
Ecto.Multi.new()
|> Ecto.Multi.update(:item, item_changeset)
|> Ecto.Multi.insert(:approval, approval_changeset)
|> MyApp.Repo.transaction()
```

- If the transaction **commits**, the held logs ship in the order they were emitted.
- If the transaction **rolls back** (a later step fails, or you call `Repo.rollback/1`), the held logs are dropped — the audit trail never claims a change happened that the database then undid.

This applies regardless of nesting depth — only the outcome of the outermost transaction decides whether held logs ship or are dropped.

#### Limitation

Schema tracking intercepts `Repo.insert`, `Repo.update`, `Repo.delete` (single-record), including when called through `Ecto.Multi.insert/update/delete`. It does **not** intercept:

- `Repo.update_all`, `Repo.delete_all`, `Ecto.Multi.insert_all/update_all/delete_all`, or raw `Ecto.Query` operations (no changeset, so no diff is possible)
- `Repo.insert_or_update` / `Ecto.Multi.insert_or_update` — Ecto resolves this internally without calling back through the repo's public `insert/2`/`update/2`, so it bypasses the override. Use an explicit `Multi.insert`/`Multi.update` (or check `changeset.data.id` yourself) if you need this tracked.

---

### 2. Controller level — declarative plug

For business events that span multiple schemas, or events with no schema change. Declare once at the top of a controller — no per-function code.

```elixir
defmodule MyAppWeb.Items.ItemApprovalController do
  use MyAppWeb, :controller

  plug AuditTrail.ControllerAudit,
    write: [
      api_approve:          "item:approved",
      api_decline:          "item:declined",
      api_request_revision: "item:revision_requested",
      api_acknowledge:      "item:revision_acknowledged"
    ]

  def api_approve(conn, %{"id" => id} = params) do
    # No AuditTrail call needed here — the plug handles it
    case ItemApproval.approve(id, params) do
      {:ok, updated}  -> json(conn, %{data: updated})
      {:error, _} = e -> respond_error(conn, e)
    end
  end
end
```

The plug hooks into `Plug.Conn.register_before_send` and fires after your action returns, capturing the final HTTP status code.

#### `write` vs `read`

| Key | Logs when |
|-----|-----------|
| `write` | Always — success **and** failure. A rejected approval is still an auditable event. |
| `read` | Only on 2xx — failed reads (404, 403) are noise and are dropped. |

#### What gets logged per event

| Field | Value |
|-------|-------|
| `event_type` | the string you provided, e.g. `"item:approved"` |
| `action` | the Phoenix action atom, e.g. `api_approve` |
| `operation` | `"write"` or `"read"` by default — `"create"`/`"update"`/`"delete"` if you use those keys (see below) |
| `resource` | `nil` unless `resource:` is set on the plug (see below) |
| `resource_id` | `nil` unless found in `conn.params["id"]`/`conn.params["uuid"]`, or `id_param:` (see below) |
| `http_status` | `200`, `422`, `403`, etc. |
| `success` | `true` / `false` |
| `params` | sanitized request params (sensitive fields stripped automatically) |

#### Tagging the resource, record, and CRUD operation (optional)

`write`/`read` control *when* an event is logged, but don't say *what* was changed or *which one*. If you want logs to carry a resource name, the specific record's id, and a specific CRUD operation (useful for filtering audit history down to e.g. "every `update` on `payment` `c9b031c6-...`"), pass `resource:` and split actions across `create:`, `update:`, `delete:` instead of (or alongside) `write:`:

```elixir
plug AuditTrail.ControllerAudit,
  resource: "payment",
  create: [api_charge: "payment:charged"],
  update: [api_refund: "payment:refunded"],
  delete: [api_void:   "payment:voided"],
  read:   [api_show:   "payment:viewed"]
```

`create`, `update`, and `delete` behave exactly like `write` (logged on success and failure) — they just set a more specific `operation` value. This is entirely optional: existing `write:`/`read:` declarations keep working unchanged, and `resource` is simply `nil` if you don't set it.

`resource_id` — the id of the specific record touched — is captured automatically from `conn.params["id"]` or `conn.params["uuid"]`, whichever is present. If your route uses a different param name, point at it with `id_param:`:

```elixir
plug AuditTrail.ControllerAudit,
  resource:  "idf",
  id_param:  "uuid",   # matches conn.params["uuid"]; also accepts an atom or a list of names
  update:    [api_update_idf: "idf:updated"]
```

With this in place you can pull the full history of one record:

```elixir
AuditTrail.get_logs(%{resource: "idf", resource_id: "c9b031c6-6d9e-494b-8399-236940ca9d10"})
```

If `conn.params` has neither the default keys nor your `id_param:`, `resource_id` is just `nil` — no error, no change to existing behavior.

#### Auto-stripped sensitive fields

The following param keys are dropped entirely before logging, regardless of what you pass:

```
password  new_password  current_password  confirm_password
code  token  access_token  refresh_token  methods_token
secret  client_secret  api_key  private_key
```

This list is `AuditTrail.Config.sensitive_fields/0` — shared by `ControllerAudit`, `AuditTrail.LiveViewAudit`, and `AuditTrail.Sanitizer` (which redacts any matching key, at any nesting depth, in *every* payload — schema-level diffs, `monitor`/`emit`'s `original_record:` diffing, everything — right before it ships). One list, one place to extend it:

```elixir
config :audit_trail, sensitive_fields: ~w(
  password token secret
  kra_pin id_number tax_payer_pin
)
```

This **replaces** the defaults rather than adding to them — include what you still want redacted alongside your own app-specific fields (national ID numbers, tax PINs, anything else that shouldn't end up in Loki/Postgres). Matching is exact and case-insensitive against the key name (`password_hash` does **not** match `password` — list the exact field name your schema uses).

`ControllerAudit`/`LiveViewAudit` *drop* the key entirely from `params`; `Sanitizer` (the backstop covering diffs, `changes`, and anything else) *replaces the value* with `"[REDACTED]"` instead of dropping the key — so a diff still shows *that* a sensitive field changed, just not to what.

#### Oversized values are truncated

File uploads and other large payloads (base64 blobs, big lists) are truncated before being logged, so a single upload action doesn't dump megabytes into the audit trail:

- Any string/binary value over 2,000 bytes becomes `"...truncated (N bytes)"`
- Any list with more than 20 items becomes `"...truncated (N items)"`

This applies recursively through nested maps/lists in `params`.

#### Login flows — set the actor before returning

Because the authentication plug runs before the login action, there is no `current_user` in `conn.assigns` at login time. Call `AuditTrail.set_actor/1` inside the success branch so the plug captures the real identity when `before_send` fires:

```elixir
plug AuditTrail.ControllerAudit,
  write: [api_login: "auth:login", api_logout: "auth:logout"]

def api_login(conn, params) do
  case Auth.login(params) do
    {:ok, user, token} ->
      AuditTrail.set_actor(user)   # ← must be called before returning
      json(conn, %{token: token})

    {:error, reason} ->
      # actor stays anonymous — failed logins are still logged
      respond_error(conn, reason)
  end
end
```

#### Plain atom shorthand

If you don't care about the event name and just want the action name (minus the `api_` prefix), pass a plain atom:

```elixir
plug AuditTrail.ControllerAudit,
  write: [:api_approve, :api_decline]
# logs as "approve" and "decline"
```

---

### 3. API / service level — manual emit

For background jobs, external API calls, or any event that doesn't go through a controller.

#### `AuditTrail.emit/2`

```elixir
AuditTrail.emit("payment:processed", %{
  amount:    order.total,
  currency:  "KES",
  gateway:   "mpesa",
  reference: txn.ref
})
```

#### Tagging the resource, record, and CRUD operation (optional)

Pass `resource:`, `resource_id:`, and `operation:` in the data map to record what kind of thing this event concerns, which specific record, and which CRUD operation it represents. All three are optional — omit them and they're simply absent (`nil`) from the payload, exactly like before this was added:

```elixir
AuditTrail.emit("payment:processed", %{
  resource:    "payment",
  resource_id: txn.id,
  operation:   "update",
  amount:      order.total,
  currency:    "KES",
  gateway:     "mpesa",
  reference:   txn.ref
})
```

There's no fixed list of operations — use whatever fits, e.g. `"create"`, `"read"`, `"update"`, `"delete"`. If you omit `resource_id:` but the data map has an `:id` or `:uuid` key, that value is used automatically.

#### Diffing against a prior record (optional) — for data outside your own Repo

`monitor/4`'s `original_record:`/`params:` diffing only helps when you have a `conn` and an Ecto result tuple. For something with **no Ecto changeset at all** — a row that lives in a library's own database, a third-party API's own resource — pass `original_record:` to `emit/2` instead:

```elixir
# New payment: no original_record, meta is exactly what you passed
AuditTrail.emit("payment:processed", %{
  resource_id: txn.ref,
  operation:   "create",
  amount:      order.total,
  currency:    "KES",
  gateway:     "mpesa",
  reference:   txn.ref
})

# Existing payment moving to a new state: original_record triggers a diff
AuditTrail.emit("payment:processed", %{
  resource_id:     txn.ref,
  original_record: previous_txn,   # a plain map or struct — whatever you have
  status:          "success",
  amount:          txn.amount
})
```

When `original_record:` is present, it's popped out of the logged payload (never dumped whole) and replaced with `meta.changes` — a field-level diff of the rest of the data map against it, reusing the exact same comparison `monitor/4` uses. `operation` defaults to `"update"` in that case (still overridable). Omit `original_record:` and behavior is identical to before this existed.

**This has no `conn`/`socket` dependency** — unlike `monitor/4`, `emit/2` never needed one, so this works unchanged from a controller, a LiveView `handle_event`, an Oban job, or a webhook handler. If you also want request context (IP, user agent, request id) attached, pass `ip_address:`/`user_agent:`/`request_id:` explicitly — extract them however your caller has them (`conn` in a controller, `socket.assigns` if you captured them at `mount/3`, nothing at all in a background job).

**Limitations worth knowing:**
- This is a *self-reported* diff — there's no changeset validating it against what the external system actually persisted, unlike `RepoWatcher`'s changeset-based diffing. It's only as honest as the `original_record` you hand it.
- Comparison is shallow/top-level per field, same as `monitor/4` — a nested map that changed is shown as one whole `from`/`to` blob, not deep-diffed.
- Field names must match between `original_record` and the rest of the data map for a field to be compared at all.
- `:status` is deliberately excluded from the "reserved, never diffed" keys, since it doubles as a very common business field name (payment/item status) as well as `emit/2`'s own log-severity hint (`"info"`/`"success"`/`"failure"`) — know that overlap exists if you rely on both meanings in the same call.

The actor is picked up from the process dictionary automatically. Override it per-event when needed:

```elixir
AuditTrail.emit("nightly:sync", %{
  actor_id:   "system:scheduler",
  actor_name: "Nightly Sync",
  records:    synced_count
})
```

#### `AuditTrail.monitor/4` — wrap a result tuple inline

```elixir
def approve(conn, %{"id" => id}) do
  original = Items.get!(id)

  Items.approve(id)
  |> AuditTrail.monitor(conn, "item:approved", original_record: original)
  |> case do
    {:ok, item}     -> json(conn, %{data: item})
    {:error, _} = e -> respond_error(conn, e)
  end
end
```

`monitor/4` returns the result unchanged so it can be piped. Available options:

| Option | Purpose |
|--------|---------|
| `original_record:` | record before the change — used to compute a before/after diff |
| `params:` | the submitted params |
| `extra_meta:` | any extra map merged into the log payload |
| `resource:` | optional resource label, e.g. `"payment"` — `nil` if omitted |

`operation` is set for you automatically (`"create"` when `original_record:` is absent, `"update"` when it's present) — no need to pass it. `resource_id` is also automatic: it's the id of the record returned in `result` (on success) or of `original_record:` (on failure) — no option needed.

The second argument only ever gets pattern-matched by shape (`%{remote_ip: ...}`, `%{req_headers: ...}`) to pull IP/user-agent/request-id, never by struct name — so passing something that isn't a `conn` (a LiveView `socket`, a plain map, `nil`) doesn't crash, it just yields `"unknown"` for those three fields while the `original_record:`/`params:` diffing still works exactly the same. If you're not in a Plug pipeline at all, `AuditTrail.emit/2` (below) is usually the better fit anyway — it never needed a `conn` in the first place.

#### `AuditTrail.log_external_api/3` — wrap an external HTTP call

```elixir
# Wrap a function — timing and success/failure captured automatically
AuditTrail.log_external_api("mpesa:stk_push", "M-Pesa", fn ->
  Mpesa.initiate_stk_push(phone, amount)
end)

# Log a result you already have
AuditTrail.log_external_api("twilio:sms", "Twilio", %{
  status:      "success",
  duration_ms: 340,
  to:          phone_number
})
```

#### `AuditTrail.set_actor/1` — set who is acting

```elixir
# From a user struct — picks up :id, :email, :name / :username automatically
AuditTrail.set_actor(user)

# For background/system actors
AuditTrail.set_actor(:system, "nightly-cleanup")
```

Stored in the **process dictionary**. Every subsequent `emit`, `monitor`, and `log_external_api` in the same process uses this identity.

> **Background jobs:** new processes (Oban jobs, `Task.async`) do not inherit the process dictionary. Call `AuditTrail.set_actor/1` at the top of your job's `perform/1`. Spawning via `AuditTrail.Task.async/start/start_link` instead of `Task`'s own propagates both the actor and tenant (see below) automatically.

#### `AuditTrail.set_tenant/1` — multi-tenant scoping

For multi-tenant apps, tag every subsequent event in the current process with an org/tenant id:

```elixir
AuditTrail.set_tenant(org.id)
```

Like `set_actor/1`, this is stored in the **process dictionary** — set it once per request (e.g. in a plug, alongside `set_actor/1`) and every `emit`/`monitor`/schema-tracked Repo call in that process picks it up automatically as a `tenant` field. On the Loki adapter it's also added as a stream label (only when set, so single-tenant apps see no change), so `AuditTrail.get_logs(%{tenant: org.id})` is a fast label-scoped query rather than a full JSON body scan. On Postgres/TimescaleDB it's a JSONB field filter, same as `resource`/`operation`.

---

### 4. Pipeline level — blanket HTTP logging

`AuditTrail.Plug` logs **every** HTTP request. This is high volume for an API-only app (one entry per request, including health checks). Only use it if you need a full HTTP access log separate from your business events.

```elixir
# router.ex
pipeline :api do
  plug :accepts, ["json"]
  plug AuditTrail.Plug   # logs every request
end
```

For most apps, `AuditTrail.ControllerAudit` on specific controllers is the better choice — it logs only the actions you explicitly care about.

#### Custom actor extractor

If your `conn.assigns` key isn't `:current_user`:

```elixir
config :audit_trail, actor_extractor: MyApp.AuditActorExtractor

defmodule MyApp.AuditActorExtractor do
  def extract(conn) do
    user = conn.assigns[:my_user]
    %{id: to_string(user.id), name: user.display_name, email: user.email}
  end
end
```

---

### 5. LiveView level — declarative `handle_event` auditing

`AuditTrail.ControllerAudit` only covers Phoenix controllers. `AuditTrail.LiveViewAudit` is the LiveView equivalent, hooking into `handle_event/3` via `Phoenix.LiveView.attach_hook/4`:

```elixir
defmodule MyAppWeb.ItemLive do
  use MyAppWeb, :live_view

  on_mount {AuditTrail.LiveViewAudit,
    resource: "item",
    write: ["approve", "decline"],
    read:  ["view_details"]}

  def handle_event("approve", %{"id" => id}, socket) do
    # No AuditTrail call needed here — the hook handles it
    ...
  end
end
```

Or attach directly (e.g. inside `mount/3`) if you don't use `on_mount`:

```elixir
def mount(_params, _session, socket) do
  {:ok, AuditTrail.LiveViewAudit.attach(socket, resource: "item", write: ["approve"])}
end
```

**This isn't identical to `ControllerAudit`.** `attach_hook` intercepts `handle_event/3` *before* your callback runs — there's no LiveView equivalent of `register_before_send` to check an outcome afterward. So every configured event is logged the moment it's dispatched, regardless of whether your `handle_event/3` clause then succeeds, raises, or no-ops. `write:`/`read:` behave identically today — the split exists so the config shape matches `ControllerAudit` and `operation` gets tagged for filtering. `resource_id` is picked up from the event params' `"id"`/`"uuid"` key (or `id_param:`), and sensitive keys are stripped, same conventions as `ControllerAudit`.

---

## Reading logs back

```elixir
# All logs (last 24 hours by default)
{:ok, logs} = AuditTrail.get_logs()

# Only a specific user's logs
{:ok, logs} = AuditTrail.get_logs(%{actor_id: "user-uuid"})

# Filter by event type
{:ok, logs} = AuditTrail.get_logs(%{type: "auth:login"})

# Filter by resource and CRUD operation (only matches logs that set these — see below)
{:ok, logs} = AuditTrail.get_logs(%{resource: "payment", operation: "update"})

# Full history of one specific record
{:ok, logs} = AuditTrail.get_logs(%{resource: "idf", resource_id: "c9b031c6-6d9e-494b-8399-236940ca9d10"})

# Date range + search term
{:ok, logs} = AuditTrail.get_logs(%{
  start_date: "2026-06-01",
  end_date:   "2026-06-12",
  search:     "alice@example.com",
  limit:      200
})
```

### Available filters

| Filter | Type | Description |
|--------|------|-------------|
| `actor_id` | string | Returns only this user's logs |
| `type` | string | Match a specific event type e.g. `"auth:login"` |
| `status` | string | `"success"` or `"failure"` |
| `resource` | string | *(optional field)* Match logs tagged with this resource, e.g. `"payment"` |
| `resource_id` | string | *(optional field)* Match logs for this one record's id, e.g. a uuid — pairs well with `resource` to get a single record's full history |
| `operation` | string | *(optional field)* Match logs tagged with this CRUD operation, e.g. `"update"` |
| `tenant` | string | *(optional field)* Match logs tagged with this tenant/org id — see `AuditTrail.set_tenant/1` |
| `search` | string | Substring match on the log line body |
| `start_date` | `"YYYY-MM-DD"` | Start of window (default: 24 hours ago — logged via `Logger.info` when defaulted) |
| `end_date` | `"YYYY-MM-DD"` | End of window (default: now) |
| `limit` | integer | Max results returned (default 100, max 500) |
| `before` | string | Cursor for the next page — see `AuditTrail.get_logs_page/1` |

String-keyed filter maps (e.g. straight from `conn.params`) are normalized automatically — `%{"resource_id" => "abc"}` and `%{resource_id: "abc"}` both work.

> **`resource`, `resource_id`, and `operation` are optional.** They only appear in the payload for events where you explicitly set them, or where `ControllerAudit` could auto-detect `resource_id` from `id`/`uuid` params. See "Tagging the resource, record, and CRUD operation" under [Controller level](#2-controller-level--declarative-plug) and [API / service level](#3-api--service-level--manual-emit). Logs written without these fields simply won't match a `resource`/`resource_id`/`operation` filter — existing code and queries are unaffected.

### Log entry structure

```elixir
%{
  timestamp: ~U[2026-06-12 10:34:21Z],
  labels: %{
    "app"  => "heart-ke",
    "type" => "auth:login",
    "user" => "550e8400-e29b-41d4-a716-446655440000"
  },
  payload: %{
    "event_id"    => "d4e5f6...",
    "event_type"  => "auth:login",
    "actor_id"    => "550e8400-...",
    "actor_name"  => "Alice Kamau",
    "actor_email" => "alice@example.com",
    "http_status" => 200,
    "success"     => true,
    "action"      => "api_login",
    "resource"    => nil,
    "resource_id" => nil,
    "operation"   => "write",
    "params"      => %{"email" => "alice@example.com"},
    "timestamp"   => "2026-06-12T10:34:21Z"
  }
}
```

`resource` and `resource_id` are `nil`, and `operation` is `"write"`/`"read"`, by default. They only carry a more specific value (e.g. `"payment"` / `"c9b031c6-..."` / `"update"`) when you opt in — see the "Tagging the resource, record, and CRUD operation" subsections above.

### Access control pattern

The reader does no access control — enforce it in your controller:

```elixir
defmodule MyAppWeb.AuditLogController do
  def index(conn, params) do
    user     = conn.assigns.current_user
    is_admin = user_is_admin?(user)

    # Non-admins see only their own logs
    filters = if is_admin, do: params, else: Map.put(params, :actor_id, to_string(user.id))

    case AuditTrail.get_logs(filters) do
      {:ok, logs} -> json(conn, %{data: logs})
      {:error, _} -> json(conn, %{data: []})  # degrade gracefully if Loki is down
    end
  end
end
```

---

## Testing

`enabled: false` (the recommended `config/test.exs` setting) silences AuditTrail entirely — good for not spamming Loki in CI, bad for asserting your code actually logs what it should. Use `AuditTrail.Adapters.Test` instead when you want that coverage:

```elixir
# config/test.exs
config :audit_trail,
  enabled:         true,
  storage_adapter: AuditTrail.Adapters.Test,
  batch_size:      1   # flush every log immediately (default flush_interval is 2000ms)
```

```elixir
defmodule MyApp.Items.ApprovalTest do
  use MyApp.DataCase, async: false   # shared in-memory table — don't run async

  setup do
    AuditTrail.Test.clear()
    :ok
  end

  test "approving an item logs an audit event" do
    {:ok, item} = Items.approve(item.id)

    AuditTrail.Test.assert_logged("item:approved", %{resource_id: item.id})
  end

  test "rejecting an item does not log an approval" do
    Items.reject(item.id)

    AuditTrail.Test.assert_not_logged("item:approved")
  end
end
```

- `assert_logged/2` and `assert_not_logged/2` match on `event_type` plus an optional map of expected fields, checked recursively — so you can assert into `meta`: `AuditTrail.Test.assert_logged("schema:update", %{meta: %{changes: %{status: %{"to" => "approved"}}}})`.
- Matching happens against the raw, atom-keyed log map built by `AuditTrail.Logger`, not the JSON-round-tripped payload you get back from `get_logs/1`.
- Logging is inherently asynchronous (buffered, flushed on a timer/batch size), so `assert_logged/2` polls for up to 200ms before failing.
- This is a single shared ETS table, not scoped per test process — run these tests with `async: false`.

---

## Loki stream labels

Every log entry is stored in Loki as a stream with three labels:

| Label | Source | Example |
|-------|--------|---------|
| `app` | `Config.app_name()` | `"heart-ke"` |
| `type` | event type | `"auth:login"` |
| `user` | actor ID | `"550e8400-..."` |

All other fields live in the JSON log line body. Loki stream labels are what make queries fast — the `actor_id` filter works because the `user` label is indexed.

> **Cardinality note:** the `user` label creates one stream per `(app, type, user)` combination. For apps with tens of thousands of users this is high cardinality. If Loki performance degrades, switch to a JSON line filter in the reader: `| json | actor_id="<uuid>"` and remove `user` from the stream labels in `Shipper.build_loki_body/1`.

---

## Crash reporting

When `crash_reporting: true` (the default), AuditTrail registers an OTP `:logger` handler that intercepts `:error`, `:critical`, `:emergency`, and `:alert` level events.

- Crash details (exception, module, stacktrace, PID, actor context) are extracted
- Crashes are deduplicated — the same exception within `dedup_window_minutes` is emailed only once
- An email is routed to the matching team via `crash_routing`
- The crash is also emitted to Loki as `event_type: "crash:error"` so it appears in Grafana

```elixir
config :audit_trail,
  crash_routing: [
    %{match: {:module, ~r/MyApp\.Payments/}, to: ["payments@company.com"]},
    %{match: :default,                       to: ["engineering@company.com"]}
  ]
```

To disable:

```elixir
config :audit_trail, crash_reporting: false
```

---

## Limitations

| Limitation | Detail |
|------------|--------|
| **Loki dependency (Loki adapter only)** | If Loki is unreachable, logs are retried up to `shipper_retries` times then dropped to in-memory dead-letter. Dead-letter does not survive a node restart. Use the Postgres or TimescaleDB adapter to avoid this entirely. |
| **No bulk operation tracking** | `Repo.update_all`, `Repo.delete_all`, `Ecto.Multi.insert_all/update_all/delete_all`, and raw `Ecto.Query` calls bypass `RepoWatcher` — there's no changeset to diff. `Repo.insert_or_update`/`Ecto.Multi.insert_or_update` also bypass it (Ecto resolves these without calling back through the repo's public `insert/2`/`update/2`). |
| **Nested-field filtering requires opting the embed/assoc in separately** | The parent's `track: [...]` only filters top-level fields. To filter *within* an embed/association, add `use AuditTrail.Schema, track: [...]` to that nested schema module too — see [Embeds and associations are diffed too](#embeds-and-associations-are-diffed-too). Embeds/associations still don't get an independent `schema:*` event of their own, since they don't go through the Repo directly. |
| **Process-local actor context** | `set_actor/1` uses the process dictionary and is not inherited by child processes. Background jobs must call it themselves. |
| **Log immutability (Loki adapter)** | Loki logs cannot be updated or deleted once written. Postgres/TimescaleDB logs can be deleted via SQL but there is no built-in API for it. |
| **Label changes not retroactive (Loki adapter)** | Adding or removing a stream label (e.g. `user`) only affects new logs. Old logs remain unfiltered by that label. |
| **Default 24h read window (Loki adapter)** | `get_logs/1` with the Loki adapter defaults to the last 24 hours. Pass `start_date` for a longer range. Postgres/TimescaleDB adapters query all rows up to `limit` if no date filters are given. |
| **Adapter migration is one-way** | Switching adapters does not migrate historical logs. Both backends can run in parallel temporarily by writing a thin adapter that fans out to both. |
| **In-memory deduplication** | Crash dedup state is in ETS and is lost on restart. |
| **`resource`/`resource_id`/`operation` are opt-in, not retroactive** | They're plain payload fields (no migration needed — `payload` is already `jsonb`/free-form). Logs written before you start setting them have `resource: nil` and won't match a `resource`/`resource_id`/`operation` filter. `resource_id` auto-detection in `ControllerAudit` only checks `id`/`uuid` params unless you set `id_param:` — a different param name won't be picked up without it. |

---

## Troubleshooting

**`UndefinedFunctionError: AuditTrail.ControllerAudit.init/1 is undefined`**

The `plug` macro calls `init/1` at compile time. Recompile with force:
```bash
mix deps.compile audit_trail --force && mix compile --force
```

**Warnings about undefined `AuditTrail.ControllerAudit` during `mix compile`**

Same root cause as above. The library beam files must exist before the host app compiles its controllers.

**Regular users see no logs (empty list)**

The `user` Loki stream label must match what was written. Logs written before the `user` label was added to the shipper won't be matched by `{user="<id>"}`. Only newly generated logs filter correctly.

**`{:already_started, #PID<...>}` on startup**

AuditTrail is listed in `application.ex` children. Remove it — it starts itself as an OTP application.

**Connection refused spam in dev logs**

Set `enabled: false` in `config/dev.exs`. The buffer accepts calls silently but the shipper never fires.

**Loki crash-loops on startup**

Check for `field buckets not found` — the S3 config key must be `bucketnames` not `buckets`. Also ensure `schema_config` is present (required in Loki 3.x even with filesystem storage).