Current section
Files
Jump to
Current section
Files
README.md
# ExRocket
[](https://github.com/mindreframer/ex_rocket/actions/workflows/elixir.yml)
[](https://github.com/mindreframer/ex_rocket/actions/workflows/release_with_caching.yml)
## About
ExRocket is an Elixir NIF backed by [RocksDB](https://github.com/facebook/rocksdb) through Rust. It provides a safe, fast API while keeping keys and values as binaries, so applications can choose their own storage format.
ExRocket is a continuation of [Rocker](https://github.com/Vonmo/rocker), a RocksDB NIF for Erlang.
> **Looking for the complete API reference?** See the [ExRocket Cheatsheet](https://github.com/mindreframer/ex_rocket/blob/main/CHEATSHEET.md) for examples covering key/value operations, column families, batches, iterators, snapshots, backups, and merge operators.
## Installation
Add `ex_rocket` to the dependencies in `mix.exs`:
```elixir
def deps do
[
{:ex_rocket, "~> 0.4"}
]
end
```
## Quick start
Open a database, write binary keys and values, and read them back:
```elixir
db_path = Path.join(System.tmp_dir!(), "my_ex_rocket_db")
{:ok, db} = ExRocket.open(db_path, %{create_if_missing: true})
:ok = ExRocket.put(db, "greeting", "hello")
{:ok, "hello"} = ExRocket.get(db, "greeting")
:undefined = ExRocket.get(db, "missing")
:ok = ExRocket.delete(db, "greeting")
```
Write several changes atomically:
```elixir
{:ok, 3} =
ExRocket.write_batch(db, [
{:put, "user:1", "alice"},
{:put, "user:2", "bob"},
{:delete, "obsolete"}
])
{:ok, [{:ok, "alice"}, {:ok, "bob"}, :undefined]} =
ExRocket.multi_get(db, ["user:1", "user:2", "missing"])
```
Atomic visibility is not the same as machine-crash durability. The default
batch path keeps the WAL enabled but does not request a synchronous filesystem
flush. ExRocket 0.5.0 adds `write_batch/3` for explicit durability boundaries:
```elixir
{:ok, 1} =
ExRocket.write_batch(
db,
[{:put, "materialization/checkpoint", "clean:42"}],
%{sync: true}
)
```
The write-option defaults are `%{sync: false, disable_wal: false}`. Do not
combine `sync: true` with `disable_wal: true`; a synchronous WAL guarantee
cannot be provided while the WAL is disabled. WAL-disabled writes are intended
only for rebuildable bulk-load data.
Applications that deliberately group non-synchronous writes can establish a
separate WAL boundary with `ExRocket.flush_wal(db, true)`. A synchronous clean
checkpoint written with `write_batch/3` is usually easier to reason about.
Use a column family to keep related data separate:
```elixir
:ok = ExRocket.create_cf(db, "sessions")
:ok = ExRocket.put_cf(db, "sessions", "token", "active")
{:ok, "active"} = ExRocket.get_cf(db, "sessions", "token")
```
Page through one stable iterator view with bounded native transfers:
```elixir
{:ok, iterator} = ExRocket.iterator(db, {:start})
{:ok, rows, status} =
ExRocket.iterator_take(iterator, %{
max_entries: 1_000,
max_bytes: 4 * 1024 * 1024
})
```
`max_entries` is required and capped at 100,000. `max_bytes` counts key/value
payload, is capped at 64 MiB, and excludes Erlang term overhead. If the first
row alone exceeds the byte limit, it is returned to guarantee progress. A
bound-first page returns `:more`; only observed exhaustion returns
`:end_of_iterator`.
Release the database path deterministically when ownership ends:
```elixir
:ok = ExRocket.close(db)
:ok = ExRocket.close(db) # idempotent
{:error, :closed} = ExRocket.get(db, "key")
```
A live iterator or snapshot keeps its native owner safe, so close returns
`{:error, :resource_busy}` until that child resource is released. Successful
close drops RocksDB and releases its filesystem lock before returning; normal
resource garbage collection remains a fallback.
See the
[full cheatsheet](https://github.com/mindreframer/ex_rocket/blob/main/CHEATSHEET.md)
for iterators, merge operators, snapshots, checkpoints, backups, and
error-handling patterns. Configuration keys and types are defined in the
[canonical option reference](OPTIONS.md); existing applications should follow
the [0.5.0 upgrade guide](UPGRADING.md).
## Technology
| Elixir module | Native crate | Rust package | Bundled RocksDB |
| --- | --- | --- | --- |
| `ExRocket` | `native/rocker` | `rust-rocksdb 0.51` | 11.1.2 |
## Supported OS
Precompiled NIFs are published for:
- Apple Silicon and Intel macOS
- ARM64 and x86-64 Linux with glibc
- ARM64 and x86-64 Linux with musl, including Alpine Linux
- x86-64 Windows
Other targets can build from source with `FORCE_BUILD=1` and the build requirements below.
## Features
- Key/value operations
- Column families
- Atomic batch writes with explicit sync/WAL options
- Exact, validated database options
- Range, prefix, and bounded bulk iterators
- Range deletion
- Multi-get
- Snapshots
- Deterministic safe database close
- Checkpoints
- Backup API
- Counter, Erlang term, and bitset merge operators
## Performance
Storage-backed NIFs run on Rustler dirty I/O schedulers so cache misses, WAL
flushes, iterator walks, and native lock waits do not block normal BEAM
schedulers. Dirty dispatch adds overhead to very small hot-cache calls; use
`iterator_take/2` to amortize NIF transitions during scans.
ExRocket's historical normal-scheduler hot-key microbenchmark measured
approximately 2.46 million reads/second and 470,000 writes/second on an Apple M3
Ultra. ADR002 benchmark reports include both storage throughput and independent
BEAM heartbeat latency. Real workloads vary with key/value size, cache state,
durability settings, compaction, concurrency, and storage hardware.
## Source build requirements
- Erlang 24 or newer
- Rust 1.91 or newer; the version is pinned in `rust-toolchain.toml`
- Clang 15 or newer
## Validation
Run `scripts/qa_check.sh` for the staged Elixir, Rust, NIF smoke, and package
quality gate. Successful dependency setup stays quiet; failures retain their
complete diagnostic output.
CI uses source-aware Mix/Cargo caches with lockfile restore keys, per-target
release caches, and no-Rust consumer jobs that compile the unpacked Hex package
and load each published native artifact through the public API.
## Release
1. Bump the version in `mix.exs` and `native/rocker/Cargo.toml`; verify parity
with `scripts/project_version.sh`.
2. Run `scripts/qa_check.sh` and push the green version commit.
3. Dispatch `Build precompiled NIFs` with `publish: false`; verify all seven
target builds and functional smoke tests.
4. Dispatch the same workflow with `publish: true`. It validates the exact
artifact set and digests in a draft before atomically publishing the tag and
GitHub release.
5. Run `mix clean && mix compile`, followed by
`mix rustler_precompiled.download ExRocket --all` locally.
6. Commit the generated checksum manifest and require all source and
no-Rust precompiled-consumer CI jobs to pass.
7. Inspect the unpacked package before publishing Hex.
## Status
Passed all functional and performance tests.
## License
ExRocket is licensed under the [Apache License 2.0](http://www.apache.org/licenses/LICENSE-2.0.html).