orchiddb

# OrchidDB for Elixir

Compile graph queries to SQL with the shared database-free compiler. Optionally
execute on a caller-owned ADBC connection, exposing the native **Arrow C Stream**
to a callback. No DuckDB or other engine is bundled. Query results stay in the
application without row or IPC conversion. Explicit statistics generation sends
bounded collection batches to the shared core through the NIF.

```elixir
# mix.exs
{:orchiddb, "~> 0.1.0"}
# Add {:adbc, "~> 0.12"} when using the Arrow execution convenience function.
```

Version 0.1.0 is published on Hex. Run the standalone example against that package:

```sh
cd examples
mix deps.get
export ORCHIDDB_NATIVE_LIBRARY=/absolute/path/liborchiddb_compiler.dylib
mix run compile.exs
mix run duckdb.exs
```

The small C NIF builds with your system compiler (`make`, Erlang headers and `cc`).
The [macOS ARM64 compiler archive](https://github.com/OrchidDB/OrchidDB-native/releases/download/v0.1.0/orchiddb-compiler-v0.1.0-aarch64-apple-darwin.tar.gz) supplies the library without a Rust build. Extract it and set the variable above to its `lib/liborchiddb_compiler.dylib`. For other platforms, build the matching compiler from
[OrchidDB-native](https://github.com/OrchidDB/OrchidDB-native) at `NATIVE_REVISION`;
its core revision must match `CORE_REVISION`. No compiler binary is downloaded
implicitly. The compiler uses a dirty CPU scheduler and its own Rust runtime.
One shared library remains loaded for the VM lifetime; switching it requires a VM restart.

```elixir
request = %{
  version: 1, dialect: "duckdb", language: "cypher",
  query: "MATCH (p:Person) RETURN p.name AS name",
  tables: [%{name: "people", columns: [
    %{name: "id", data_type: "int64"}, %{name: "name", data_type: "string"}]}],
  nodes: [%{label: "Person", table: "people", id: "id", properties: %{name: "name"}}]
}
{:ok, plan} = OrchidDB.compile(request)
# conn and destination are your configured ADBC connections.
OrchidDB.query_arrow(conn, request, fn arrow_stream ->
  Adbc.Connection.bulk_insert!(destination, arrow_stream, table: "result")
end)
```

The stream can be consumed once and is valid **only inside the callback**.
Do not return or retain its pointer. ADBC owns statement/stream cleanup; OrchidDB
never stops your connection or database process or alters transactions. Register
extensions/UDFs yourself, declaring matching function signatures in the request.
See [ADBC query_pointer](https://adbc.hexdocs.pm/Adbc.Connection.html#query_pointer/5).

Requests support `cypher`, `gremlin`, and `sparql`; mapping, schema, ontology and
function signatures follow core's JSON v1 compiler contract. DuckDB and PostgreSQL
SQL rendering and mixed-engine execution through `query_federated` are supported.
Gremlin text compiles to SQL; there is no native Gremlin executor.
Parameters are literals specialized into SQL; compile again after changing them.
Errors return `{:error, reason}`. The integration suite uses the real DuckDB ADBC driver to execute compiled graph
queries and ingest Arrow C Streams directly. It verifies int64 values, nulls,
caller transactions/rollback, connection reuse, and consumer-error cleanup.
Run `mix test` to download the DuckDB test driver and exercise the complete path.

## Permission pushdown

Add a flat effective-grants table to the request's `tables`, then attach one or more scopes to a node. Each scope matches a relation resource ID against a node source column; all scopes must contain effective grants for the query principal.

```elixir
alias OrchidDB.Permission

grants = Permission.relation("effective_grants", "document", "view")
project_grants = Permission.relation("effective_grants", "project", "view")
request = Map.merge(request, %{
  authorization: Permission.authorization("user", "alice"),
  nodes: [%{label: "Document", table: "documents", id: "id",
    properties: %{title: "title", project_id: "project_id"},
    permission_scopes: [
      Permission.scope("id", grants),
      Permission.scope("project_id", project_grants)
    ]}]
})
{:ok, plan} = OrchidDB.compile(request)
```

Use `Permission.relation/4` to override relation column names. The compiler fails closed when scopes have no principal; it does not connect to or synchronize an external permission system.

## Release

`mix hex.build` creates the source package including the C NIF. Workflow
`release.yml` validates the matching version tag and publishes with the repository
secret `HEX_API_KEY` in environment `hex`. Hex package `orchiddb` version 0.1.0 is published. Compiler binaries are independently
released from OrchidDB-native and pinned here.

[GPL-3.0-only license](LICENSE.md).

## One-time statistics

```elixir
{:ok, statistics} = OrchidDB.Statistics.generate(request, fn work ->
  MySession.collect_bounded(connection, work)
end)
{:ok, plan} = OrchidDB.compile(request, statistics: statistics)
:ok = OrchidDB.Statistics.save(statistics, "statistics.json")
{:ok, _} = OrchidDB.Statistics.clear(statistics)
{:ok, statistics} = OrchidDB.Statistics.load("statistics.json")
```

The callback executes collection SQL on the application's existing session. It
must enforce the supplied `timeout_ms`, `max_rows`, and `max_bytes`, returning
`{:ok, %{"rows" => rows}}` or `{:ok, %{"ipc" => base64_arrow_stream}}`. Return
`{:error, reason}` if bounded execution is unavailable. ADBC's current Elixir API
does not expose query interruption, so this client does not claim that a BEAM
call timeout bounds database execution; use an adapter with genuine cancellation
or a server-side statement deadline.

Generation may make several calls. Shared Rust code chooses requests and computes
statistics; the Elixir client only transports them. `statistics.report` records
coverage and skipped sources. A retained native catalog is reused by compilation,
which preserves all diagnostics and never reads data or refreshes automatically.
Pass `statistics: statistics` to `query_arrow` as well. Save/load persists portable
snapshots rather than process-local handles. Release with `clear` after the last
user finishes. To regenerate, obtain a replacement successfully before clearing
the old handle. `OrchidDB.statistics_command/2` exposes the shared protocol for
batch submission and explicit cancellation.

Collectors must set `truncated: true` when transport limits stop a response before
EOF; the coordinator retains those observations as a partial sample. It must not
infer a complete source row count from a shortened response.