ex_codecs

# Understanding Spatial Codecs

This guide is a **domain primer** for ExCodecs' spatial category: what the
data is, what each Elixir type means, which file formats you encode to, and
when to use each piece. It is not a rendering tutorial - ExCodecs only moves
spatial data in and out of interchange formats.

## What problem does this solve?

You have 3D samples from a scanner, photogrammetry pipeline, or radiance-field
export, and you need to:

1. Represent them as Elixir structs in your BEAM process
2. Write them to a portable file (or bytes)
3. Read them back later - often into another tool that expects PLY or a compact
   binary container

Spatial codecs are the encode/decode layer for that. They do **not** draw
pixels, build meshes, or run a viewer.

## Point clouds vs Gaussian splats

These are two different ways to describe a scene as **many independent
samples**. Neither is a polygon mesh (no faces, no connectivity).

### Point cloud - a bag of dots

A **point cloud** is a list of positions in space. Each point is usually just
`(x, y, z)`. Optionally it can also carry:

- a color (what the scanner/camera saw)
- a surface normal (which way the surface faced)
- arbitrary attributes (intensity, classification, …)

Think of LiDAR returns, depth-camera frames, or the corners of a scanned room:
lots of floating dots, not triangles.

```text
        ·  ·
     ·     ·  ·
   ·   ·         ·     ← each · is one Point {x,y,z[,color,…]}
        ·    ·
           ·
```

In ExCodecs that bag is a `PointCloud` (the collection) of `Point` structs.

### Gaussian splat - a bag of fuzzy ellipsoids

A **3D Gaussian splat** (often from *3D Gaussian Splatting* research) is also
a sample in space, but instead of a hard dot it is a small **oriented blob**:

- **position** - where the blob is centered
- **scale** - how large it is along its local axes (an ellipsoid, not a sphere)
- **rotation** - how that ellipsoid is oriented (a quaternion)
- **opacity** - how solid / transparent it is
- **color** - base RGB (often the “DC” spherical-harmonic color)
- **SH** - optional higher-order **spherical harmonic** coefficients that
  encode view-dependent appearance (fancy lighting detail for renderers)

```text
        (~~)           ← one Gaussian: soft blob, not a hard vertex
     (~~~~)  (~~)
        (~~~~)
```

A renderer turns millions of these blobs into an image. **ExCodecs does not
render them** - it only stores and reloads the parameters so another program
(or a future library) can.

In ExCodecs that bag is a `GaussianCloud` of `Gaussian` structs.

| | Point cloud | Gaussian splat |
|--|-------------|----------------|
| Primitive | Hard sample (dot) | Soft ellipsoid (blob) |
| Typical source | LiDAR, depth cameras, photogrammetry points | Trained / exported radiance-field scenes |
| Looks like geometry? | Sparse surface samples | Appearance-oriented volume of blobs |
| ExCodecs types | `Point` / `PointCloud` | `Gaussian` / `GaussianCloud` |

## Elixir-first, with optional Rust acceleration

Spatial encode/decode is **Elixir-first**: every format has a complete
pure-Elixir implementation, so the API works on any BEAM without a native
backend. Since **v0.2.3**, an optional Rust NIF accelerator is loaded when
the precompiled native library is available:

- **Chunked Rust pack/unpack** for EXCP and GSPL (byte-compatible with the
  Elixir path; verified by property tests).
- **mmap-backed file `stream_decode`** for EXCP/GSPL/PLY, and **binary PLY
  body unpack**.
- **Chunked `stream_encode_to_file`** for EXCP/GSPL.

Compression codecs (`:zstd`, `:lz4`, …) always use Rust NIFs; spatial formats
use the Elixir path unless acceleration is available. Pass `accel: false` to
force pure Elixir for a given call.

A common pattern is:

```elixir
{:ok, ply} = ExCodecs.Spatial.encode(cloud, format: :ply)
{:ok, packed} = ExCodecs.encode(:zstd, ply)   # NIF compresses the bytes
```

Spatial work stays on the BEAM (or the DirtyCpu NIF pool when accelerating);
you optionally compress the resulting binary with the registry codecs.

### Streaming (v0.2.3+)

EXCP / GSPL / PLY **files** decode record-by-record from disk (bounded
memory; mmap + Rust unpack when available). In-memory binaries use chunked
Rust unpack when the NIF is loaded, otherwise materialize through `decode/2`.
`stream_encode/2` still collects then encodes; use `encode_to_file/3` with an
explicit `:schema` for EXCP/GSPL incremental writes. Prefer explicit
`source: :file` or `source: :binary`. See `ExCodecs.Spatial.Stream` for
details.

## Domain types - what each one is for

```text
Point ──┐
        ├──► PointCloud ──► encode as :ply or :spatial_binary
Bounds ─┘         │
Metadata ─────────┘
Transform ────────┘   (optional pose of the whole cloud; not applied here)

Gaussian ──┐
           ├──► GaussianCloud ──► encode as :ply or :gsplat
Bounds / Metadata / Transform ─┘
```

### `Point` - one sample

One location in 3D, optionally colored / with a normal / with attributes.

```elixir
alias ExCodecs.Spatial.Point

Point.new(1.0, 2.0, 0.5,
  color: {255, 128, 0},
  normal: {0.0, 0.0, 1.0},
  attributes: %{"intensity" => 0.82}
)
```

**Use when:** you are building or iterating individual samples.

### `PointCloud` - the whole set (not a polygon)

A **collection** of points plus bookkeeping. It is **not** a mesh and **not**
a polygon. There are no edges or faces - only the list of points.

```elixir
alias ExCodecs.Spatial.{Point, PointCloud}

cloud =
  PointCloud.new([
    Point.new(0.0, 0.0, 0.0, color: {255, 0, 0}),
    Point.new(1.0, 0.0, 0.0, color: {0, 255, 0}),
    Point.new(0.0, 1.0, 0.0, color: {0, 0, 255})
  ])
```

`PointCloud` may also hold:

- `bounds` - axis-aligned box around the points (see below)
- `metadata` - free-form notes / comments
- `transform` - optional pose of the *entire* cloud (see below)

**Use when:** this is your unit of encode/decode for point data.

### `Bounds` - “how big is this cloud?”

An **axis-aligned bounding box** (AABB): min/max corners in XYZ. Useful for
culling, indexing, or quick spatial queries. It describes the extent of the
data; it is not the data itself.

```elixir
alias ExCodecs.Spatial.Bounds

Bounds.new({0.0, 0.0, 0.0}, {1.0, 1.0, 0.5})
```

**Use when:** you care about the cloud’s spatial footprint. Many codecs
recompute or omit bounds on the wire; keep them in the struct for app logic.

### `Gaussian` - one splat

One oriented ellipsoid with appearance parameters. The rotation/scale here
belong to **this splat**, not to a whole-scene pose.

```elixir
alias ExCodecs.Spatial.Gaussian

Gaussian.new({1.0, 2.0, 3.0},
  scale: {0.05, 0.02, 0.05},
  rotation: {1.0, 0.0, 0.0, 0.0},  # identity quaternion {w,x,y,z}
  opacity: 0.9,
  color: {0.8, 0.4, 0.1}
)
```

**Use when:** importing/exporting Gaussian-splat datasets.

### `GaussianCloud` - the whole splat set

Same idea as `PointCloud`, but for Gaussians.

**Use when:** this is your unit of encode/decode for splat data.

### `Transform` - pose of the *whole* cloud (not a splat)

A similarity transform (translation + quaternion rotation + **uniform** scale)
stored as **metadata**. ExCodecs does **not** apply it to points/Gaussians.

This is different from a Gaussian’s per-splat rotation/scale:

| | `Gaussian` rotation/scale | `Transform` |
|--|---------------------------|-------------|
| Applies to | One splat’s ellipsoid | The entire cloud (if you apply it) |
| Scale | Anisotropic `{sx,sy,sz}` | Uniform float |
| Applied by ExCodecs? | Stored as splat params | Stored only; you apply it |

**Use when:** a format or pipeline records “this cloud was captured with
sensor pose X” and you want to keep that alongside the samples.

### `Metadata` - comments and leftovers

String comments and a free-form map for application keys that are not geometry.

**Use when:** you need PLY header comments or app-specific tags to round-trip.

## Formats glossary - names that look similar but mean different things

Three layers of vocabulary get mixed up. Keep them separate:

| Kind | Examples | What it is |
|------|----------|------------|
| **Domain concept** | point cloud, **splat** / Gaussian splat | The *kind of 3D data* in memory |
| **Format atom** | `:ply`, `:spatial_binary`, `:gsplat` | What you pass to `ExCodecs.Spatial.encode/decode` |
| **On-wire magic / file type** | PLY text/binary, **`EXCP`**, **`GSPL`** | Bytes on disk / on the network |

```text
In memory                         On the wire
─────────                         ───────────
PointCloud      ──format: :ply──►           *.ply  (PLY header + body)
PointCloud      ──format: :spatial_binary──► magic "EXCP" …
GaussianCloud   ──format: :ply──►           *.ply  (Gaussian property names)
GaussianCloud   ──format: :gsplat──►        magic "GSPL" …
```

### “Splat” (domain concept - not a format)

**Splat** is short for **Gaussian splat**: one soft 3D ellipsoid sample (see
above). A **splat cloud** is many of them (`GaussianCloud`).

There is **no** `format: :splat`. To store splats you choose either:

- `:ply` - portable interchange (Gaussian PLY properties), or
- `:gsplat` - ExCodecs’ compact binary (on-wire magic **`GSPL`**)

### PLY / `format: :ply` - the portable interchange format

**PLY** (Polygon File Format / Stanford Triangle Format) is a **decades-old,
widely supported** 3D file format. A PLY file always starts with a readable
header, then a body of per-vertex (or per-face) properties.

Despite “polygon” in the name, PLY is routinely used as a **point list** -
just vertices with `x y z` (and maybe colors), no faces at all. Research and
Gaussian-splat tools also stuff splat fields into the same PLY property
mechanism (`f_dc_*`, `opacity`, `scale_*`, `rot_*`, `f_rest_*`).

**What it’s for**

- Hand files to MeshLab, CloudCompare, Blender, Python/`open3d`, research code
- Debug by opening ASCII PLY in a text editor
- Exchange with people / tools that have never heard of ExCodecs

**What you write in Elixir**

```elixir
ExCodecs.Spatial.encode(cloud, format: :ply)
ExCodecs.Spatial.encode(cloud, format: :ply, ply_format: :binary_le)
ExCodecs.Spatial.encode(gaussians, format: :ply)  # Gaussian property names
```

**Trade-offs**

- Universal tooling, human-inspectable (ASCII)
- Larger / slower than EXCP/GSPL; schema is flexible but verbose
- Works for **both** point clouds and Gaussian clouds

PLY options (after `format: :ply`):

- `:ply_format` or `:format` - `:ascii` (default), `:binary` / `:binary_le`,
  or `:binary_be` (**PLY body encoding**, not the Spatial format atom)
- `:comments` - header comments
- `:as` - on decode: `:auto`, `:point_cloud`, or `:gaussian_cloud`

Tiny ASCII PLY (three colored points):

```text
ply
format ascii 1.0
element vertex 3
property float x
property float y
property float z
property uchar red
property uchar green
property uchar blue
end_header
0 0 0 255 0 0
1 0 0 0 255 0
0 1 0 0 0 255
```

### EXCP / `format: :spatial_binary` - compact point-cloud bytes

**EXCP** is ExCodecs’ **own** little-endian binary container for point clouds.
The name is the 4-byte magic at the start of the payload (`"EXCP"`).

**`:spatial_binary`** is the **Elixir format atom** you pass to the API.
Same thing, two names:

| You say in code | Bytes begin with | Holds |
|-----------------|------------------|-------|
| `format: :spatial_binary` | `EXCP` | `PointCloud` only |

**What it’s for**

- Store or ship point clouds **inside your own system** (DB blob, message,
  object store) when PLY would be wasteful
- Fast encode/decode with a fixed record layout (xyz + optional color/alpha/normals)
- Pair with `:zstd` / `:lz4` afterward if you want compression

**What it’s not for**

- Giving files to MeshLab / Blender (they don’t speak EXCP)
- Storing Gaussian splats (use `:gsplat` or Gaussian `:ply`)
- Carrying arbitrary per-point string attributes, cloud metadata, or transforms
  on the wire (v1 drops those - see [Spatial wire formats](../docs/spatial_formats.md))

```elixir
{:ok, excp} = ExCodecs.Spatial.encode(point_cloud, format: :spatial_binary)
# excp starts with <<"EXCP", ...>>
{:ok, cloud} = ExCodecs.Spatial.decode(excp, format: :spatial_binary)
```

### GSPL / `format: :gsplat` - compact Gaussian-splat bytes

**GSPL** is ExCodecs’ **own** little-endian binary container for Gaussian
clouds. Magic bytes: `"GSPL"`.

**`:gsplat`** is the format atom. Again: atom in Elixir, magic on the wire.

| You say in code | Bytes begin with | Holds |
|-----------------|------------------|-------|
| `format: :gsplat` | `GSPL` | `GaussianCloud` only |

**What it’s for**

- Compact storage/transfer of splat parameters (position, DC color, opacity,
  scale, rotation, optional SH rest coefficients)
- Internal pipelines where PLY Gaussian dumps are too fat

**What it’s not for**

- Point clouds (use `:spatial_binary` or `:ply`)
- Universal desktop 3D tools (use Gaussian `:ply` for interchange)
- Preserving arbitrary per-Gaussian metadata maps on the wire (v1 drops them)

```elixir
{:ok, gspl} = ExCodecs.Spatial.encode(gaussian_cloud, format: :gsplat)
# gspl starts with <<"GSPL", ...>>
{:ok, cloud} = ExCodecs.Spatial.decode(gspl, format: :gsplat)
```

### Quick chooser

| Goal | Use |
|------|-----|
| Share points with other 3D software | `:ply` |
| Share Gaussians with research / splat tools | `:ply` (Gaussian properties) |
| Compact points inside *your* stack | `:spatial_binary` → optional `:zstd` |
| Compact Gaussians inside *your* stack | `:gsplat` → optional `:zstd` |
| “I heard splat / EXCP / GSPL” | splat = data kind; EXCP/GSPL = our binary containers |

Wire layouts and schema rules are frozen in
[Spatial wire formats](../docs/spatial_formats.md). Streaming behavior is
described in the [Elixir-first, with optional Rust acceleration](#elixir-first-with-optional-rust-acceleration)
section above and in `ExCodecs.Spatial.Stream`.

## Why a separate `ExCodecs.Spatial` API?

Compression codecs map **binary → binary** (`ExCodecs.encode(:zstd, bin)`).
Spatial codecs map **structs ↔ formats**. Overloading one function for both
shapes is confusing, so spatial work goes through `ExCodecs.Spatial`.

Both categories share the catalog (`ExCodecs.available_codecs(:spatial)`),
tagged `{:ok, _}` / `{:error, %ExCodecs.Error{}}` results, and discovery
metadata. Calling `ExCodecs.encode(:ply, …)` intentionally fails with guidance
to use the spatial API.

## End-to-end examples

### Point cloud → PLY → back

```elixir
alias ExCodecs.Spatial.{Point, PointCloud}

cloud =
  PointCloud.new([
    Point.new(0.0, 0.0, 0.0, color: {255, 0, 0}),
    Point.new(1.0, 1.0, 0.0, color: {0, 255, 0})
  ])

{:ok, ply} = ExCodecs.Spatial.encode(cloud, format: :ply)
{:ok, decoded} = ExCodecs.Spatial.decode(ply, format: :ply)

ExCodecs.Spatial.available_formats()
ExCodecs.codec_info(:ply)
```

### Compact binary for points, then compress

```elixir
{:ok, excp} = ExCodecs.Spatial.encode(decoded, format: :spatial_binary)
{:ok, packed} = ExCodecs.encode(:zstd, excp)
{:ok, excp2} = ExCodecs.decode(:zstd, packed)
{:ok, ^decoded} = ExCodecs.Spatial.decode(excp2, format: :spatial_binary)
```

### Load a sample from the package

```elixir
path = Application.app_dir(:ex_codecs, "priv/examples/spatial/cube_corners.ply")
{:ok, cloud} = ExCodecs.Spatial.decode(File.read!(path), format: :ply)

path = Application.app_dir(:ex_codecs, "priv/examples/spatial/two_gaussians.ply")
{:ok, gaussians} = ExCodecs.Spatial.decode(File.read!(path), format: :ply)
```

## Out of scope

ExCodecs does **not** provide:

- a viewer / rasterizer for points or Gaussians
- mesh topology (faces, edges)
- spatial indexes, LOD, or true progressive streaming
- supercompressed containers such as SOG

Those belong in higher-level libraries built on top of these codecs.