provenance

# Provenance

Universal artifact lineage tracking for Elixir. Tracks what code produced what data,
across modules, queries, requests, and jobs.

Every artifact gets a deterministic `kind:identifier` ID. An ETS-backed graph store
records producer/consumer relationships. A compile tracer automatically discovers
modules and classifies them into architectural layers.

Designed for use with [Sentinel](https://github.com/fun-fx/sentinel) but works
standalone in any Elixir project.

## Installation

Add to your `mix.exs`:

```elixir
def deps do
  [
    {:provenance, "~> 0.1"},
    {:ecto, "~> 3.10", optional: true},
    {:plug, "~> 1.15", optional: true}
  ]
end
```

## Quick Start

### Registering Modules

Modules can be registered manually or discovered automatically via the compile tracer.

```elixir
Provenance.register_module(MyApp.Orders,
  source_file: "lib/my_app/orders.ex",
  layer: :context,
  dependencies: ["mod:MyApp.Repo"]
)

Provenance.register_module(MyApp.Orders.Order,
  source_file: "lib/my_app/orders/order.ex",
  layer: :schema
)

Provenance.lookup("mod:MyApp.Orders")
# => {:ok, %{id: "mod:MyApp.Orders", kind: :mod, layer: :context, ...}}
```

### Recording Provenance

Record that an artifact was produced by another:

```elixir
Provenance.record("rec:orders:42", origin: "fn:MyApp.Orders.create/1")
Provenance.record("rec:order_items:99", origin: "fn:MyApp.Orders.add_item/2")
```

### Querying Related Artifacts

```elixir
Provenance.related("fn:MyApp.Orders.create/1")
# => ["rec:orders:42"]

Provenance.related("mod:MyApp.Orders")
# => ["mod:MyApp.Repo"]

Provenance.module_graph()
# => %{"mod:MyApp.Orders" => ["mod:MyApp.Repo"], ...}
```

### Process Context

Attach provenance context to the current process. Useful for tracing requests,
jobs, or any unit of work:

```elixir
Provenance.put_context(request_id: "req:F1a2b3c4d5", user_id: "usr:42")
Provenance.get_context()
# => %{request_id: "req:F1a2b3c4d5", user_id: "usr:42"}

# Context merges -- existing keys are preserved, new ones added
Provenance.put_context(trace_id: "trc:abc")
Provenance.get_context()
# => %{request_id: "req:F1a2b3c4d5", user_id: "usr:42", trace_id: "trc:abc"}
```

## Artifact ID Format

All IDs follow the `kind:identifier` scheme:

| Kind | Format | Example |
|------|--------|---------|
| `mod` | `mod:<Module>` | `mod:MyApp.Orders` |
| `fn` | `fn:<Module>.<function>/<arity>` | `fn:MyApp.Orders.get_order/1` |
| `tbl` | `tbl:<table>` | `tbl:orders` |
| `col` | `col:<table>.<column>` | `col:orders.total_cents` |
| `rec` | `rec:<table>:<pk>` | `rec:orders:42` |
| `req` | `req:<request_id>` | `req:F1a2b3c4d5` |
| `mig` | `mig:<migration_id>` | `mig:20240315_add_status` |
| `job` | `job:<queue>:<job_id>` | `job:default:j-123` |
| `cfg` | `cfg:<config_path>` | `cfg:app.database.pool_size` |

ID helpers:

```elixir
Provenance.module_id(MyApp.Orders)       # => "mod:MyApp.Orders"
Provenance.function_id(MyApp.Orders, :get_order, 1)  # => "fn:MyApp.Orders.get_order/1"
Provenance.table_id("orders")            # => "tbl:orders"
Provenance.record_id("orders", 42)       # => "rec:orders:42"
```

## Compile Tracer

Automatically registers modules and records inter-module dependencies at compile time.

Enable in `mix.exs`:

```elixir
def project do
  [
    # ...
    compilers: Mix.compilers(),
    tracers: [Provenance.CompileTracer]
  ]
end
```

Or in config:

```elixir
config :elixir, :tracers, [Provenance.CompileTracer]
```

The tracer classifies each module into an architectural layer based on file path
and module characteristics:

| Layer | Detection |
|-------|-----------|
| `:test` | Path contains `test/` |
| `:controller` | Path contains `_web/controllers/` |
| `:view` | Path contains `_web/live/` or `_web/components/` |
| `:web` | Path contains `_web/` |
| `:worker` | Path contains `/workers/` |
| `:migration` | Path contains `/migrations/` |
| `:repo` | Module name ends with `Repo` |
| `:schema` | Module exports `__schema__/1` (Ecto) |
| `:context` | Shallow lib path (depth <= 2) |
| `:lib` | Everything else |

## Architecture

```
+---------------------+
|   Your Application  |
+---------------------+
         |
    register / record / query
         |
+---------------------+     +------------------------+
|    Provenance API    |---->|  Provenance.Store      |
|  (lib/provenance.ex) |     |  (ETS GenServer)       |
+---------------------+     |                        |
                             |  :provenance_artifacts |
+---------------------+     |  :provenance_edges     |
| CompileTracer        |---->|  :provenance_modules   |
| (compile-time hook)  |     +------------------------+
+---------------------+

Process Dictionary
+---------------------+
| put_context/1        |  Per-process provenance context
| get_context/0        |  (:provenance_context key)
+---------------------+
```

- **Provenance** -- Public API. Delegates to `Provenance.Store` for persistence
  and provides ID builder functions.
- **Provenance.Store** -- GenServer owning three ETS tables: artifacts (set),
  edges (duplicate_bag, bidirectional), and modules (set).
- **Provenance.CompileTracer** -- Elixir compiler tracer. On each compiled module,
  registers it with layer classification and records remote function call edges.
- **Process context** -- Stored in the process dictionary. Designed for inclusion
  in provenance recordings during request/job execution.

## Ecto Integration (Planned)

Ecto is an optional dependency. Current support is limited to detecting Ecto schemas
in the compile tracer for layer classification. Future versions will add:

- Automatic query-level provenance via Ecto telemetry
- Schema field to artifact ID mapping
- Migration tracking

## API Reference

### Core

| Function | Description |
|----------|-------------|
| `Provenance.register_module(module, opts)` | Register a module with metadata and dependencies |
| `Provenance.record(artifact_id, opts)` | Record provenance (`:origin` option for producer) |
| `Provenance.lookup(artifact_id)` | Look up artifact info |
| `Provenance.related(artifact_id)` | List related artifact IDs |
| `Provenance.module_graph()` | Full module dependency graph |

### ID Builders

| Function | Description |
|----------|-------------|
| `Provenance.module_id(module)` | `"mod:Module.Name"` |
| `Provenance.function_id(mod, fun, arity)` | `"fn:Mod.fun/arity"` |
| `Provenance.table_id(name)` | `"tbl:name"` |
| `Provenance.record_id(table, pk)` | `"rec:table:pk"` |

### Process Context

| Function | Description |
|----------|-------------|
| `Provenance.put_context(keyword)` | Merge attrs into process context |
| `Provenance.get_context()` | Get current process context map |

## License

MIT -- see [LICENSE](LICENSE).