Current section
Files
Jump to
Current section
Files
README.md
# Regc
Regc is an OCI and Docker registry library with strict content verification.
It provides reusable clients, manifest and blob reads and writes, registry
listings, authentication, and streamed transfers. Its image helpers can check
whether a tag points to valid manifest content without pulling image layers,
resolve an exact or best-compatible platform, and retrieve a verified image
configuration. Backend-independent copy helpers transfer complete image and
artifact graphs between registries and local OCI image layouts.
## Installation
Add `regc` to the dependencies in `mix.exs`:
```elixir
def deps do
[
{:regc, "~> 0.0.1"}
]
end
```
## Reusable registry client
Create one client and reuse it across concurrent registry operations:
```elixir
client =
Regc.Client.new(
timeout: 15_000,
max_retries: 2
)
{:ok, manifest} =
Regc.Manifest.get(client, "registry.example.com/team/app:1.2.3")
{:ok, metadata} =
Regc.Manifest.head(
client,
"registry.example.com/team/app:1.2.3",
require_digest: true
)
# Release cached tokens and idle connections when the client is no longer used.
Regc.Client.close(client)
```
The built-in transport pools persistent HTTP/1.1 connections per client and
host. Fully consumed blob streams return their connection to the pool; closing
or abandoning a partial stream discards it safely.
### Credentials and Docker configuration
Provide host-keyed Basic, refresh-token, or direct registry-token credentials:
```elixir
client =
Regc.Client.new(
credentials: %{
"registry.example.com" =>
Regc.Credential.new(username: "user", password: "secret"),
"another-registry.example.com" => {:refresh_token, refresh_token}
}
)
```
A credential provider can also be an arity-one function, a module implementing
`fetch/1`, or `{module, state}` implementing `fetch/2`. Token responses are
cached by realm, service, account, scope, and credential identity. Read
operations request pull scope; write operations request pull and push scope.
Inline credentials from Docker's `config.json` can be loaded explicitly:
```elixir
{:ok, credentials} = Regc.Credential.DockerConfig.load()
client = Regc.Client.new(credentials: credentials)
```
External Docker credential helpers are not executed by this adapter.
### Host routing and TLS
Host configuration supports aliases, Distribution API path prefixes, mirrors,
custom CAs, mutual TLS, and per-endpoint request limits:
```elixir
client =
Regc.Client.new(
hosts: %{
"registry.example.com" => %{
hostname: "registry.internal:5443",
path_prefix: "distribution",
ca_certfile: "/etc/ssl/private-registry-ca.pem",
client_certfile: "/etc/ssl/regc-client.pem",
client_keyfile: "/etc/ssl/regc-client-key.pem",
mirrors: ["mirror.example.com"],
request_concurrency: 4,
request_rate: 10
},
"mirror.example.com" => %{
hostname: "mirror.internal:5443",
priority: 10
}
}
)
```
Content mirrors are attempted by descending priority before the upstream.
Authentication is resolved independently for every mirror, and sensitive
headers are never copied between endpoints. Use `tls: :insecure` only for an
explicit verification bypass, or `tls: :disabled` for a configured plaintext
endpoint.
When resolving an index with `Manifest.head/3`, strict exact matching remains
the default. Set `platform_match: :compatible_best` to use regclient-compatible
OS, architecture, CPU-variant, and Windows-version ranking:
```elixir
{:ok, metadata} =
Regc.Manifest.head(client, image,
platform: "linux/amd64/v3",
platform_match: :compatible_best
)
```
OCI references and descriptors support SHA-256, SHA-384, and SHA-512 digests.
`Regc.Blob.get/4` returns a streaming, incrementally verified
`Regc.Blob.Reader`. Always consume the reader to EOF or close it explicitly.
`Regc.Blob.head/4` reads blob metadata without downloading the body when the
registry supports `HEAD`.
Tag and catalog operations collect all pages by default. Supplying
`:page_size` returns a lazy page unless `all: true` is also set:
```elixir
{:ok, tags} = Regc.Tag.list(client, "registry.example.com/team/app")
{:ok, page} =
Regc.Tag.list(client, "registry.example.com/team/app", page_size: 100)
{:ok, next_page} = Regc.Page.next(client, page)
{:ok, repositories} =
Regc.Repository.list(client, "registry.example.com")
```
Pagination follows only validated same-origin `Link` targets and rejects
loops. `:max_pages` and `:max_page_bytes` bound automatic traversal.
Referrers can be filtered by OCI artifact type. If the registry does not
support the Referrers API, Regc uses the OCI digest-tag fallback convention:
```elixir
{:ok, descriptors} =
Regc.Referrer.list(client, subject_reference,
artifact_type: "application/vnd.example.sbom"
)
```
### Registry writes
Manifest uploads preserve exact bytes and verify any requested, descriptor,
or response digest:
```elixir
{:ok, metadata} =
Regc.Manifest.put(client, "registry.example.com/team/app:release", raw_manifest,
media_type: "application/vnd.oci.image.manifest.v1+json"
)
:ok =
Regc.Manifest.delete(
client,
"registry.example.com/team/app@#{metadata.digest}"
)
```
Blob uploads accept replayable binaries and iodata, one-shot streams, or
replayable producer functions. When a digest is known, Regc first attempts an
anonymous mount, uses a monolithic upload when appropriate, and falls back to
a fresh chunked upload for replayable sources:
```elixir
{:ok, descriptor} =
Regc.Blob.put(
client,
"registry.example.com/team/app",
{:stream, source_stream},
chunk_size: 4 * 1024 * 1024,
progress: fn event -> send(progress_process, event) end
)
:ok =
Regc.Blob.mount(
client,
"registry.example.com/team/target",
"registry.example.com/team/source",
descriptor
)
```
For explicit resumable control, `Regc.Blob.Upload` exposes upload start,
status, chunk, finalize, and cancel operations. Upload locations are
revalidated whenever a registry rotates them, and cross-origin credentials
are isolated.
`Regc.Tag.delete/3` deletes only the selected tag. It tries the native tag
delete operation first and, by default, uses a temporary replacement manifest
when compatibility fallback is needed.
## Images, artifacts, and OCI layouts
`Regc.Image.copy/4` recursively verifies and copies indexes, manifests,
configs, layers, and OCI artifact payloads. Blob existence checks,
cross-repository mounts, digest deduplication, bounded concurrency, progress
events, and cooperative cancellation avoid unnecessary or unsafe transfers:
```elixir
:ok =
Regc.Image.copy(
client,
"registry.example.com/team/source:release",
"registry.example.com/team/target:release",
platforms: ["linux/amd64", "linux/arm64"],
include_referrers: true,
include_digest_tags: true,
concurrency: 4,
progress: fn event -> send(progress_process, event) end,
cancel: fn -> Process.get(:cancel_copy, false) end
)
```
Platform filters recursively prune indexes and produce a new consistent root
index. Unfiltered manifests retain their exact source bytes. Referrers whose
subject digest changed because of filtering are not copied, since they would
no longer describe the target content.
Use `Regc.Image.retag/4` for a manifest-only tag in one repository. It does not
transfer layers or child manifests.
The `ocidir://` backend stores standard OCI image layouts with content under
`blobs/<algorithm>/<digest>` and root descriptors in `index.json`. Copy works
in every registry/layout direction. Filesystem-oriented wrappers accept a
plain directory path:
```elixir
:ok =
Regc.Image.export(
client,
"registry.example.com/team/app:release",
"/var/lib/images/app",
layout_tag: "release"
)
:ok =
Regc.Image.import(
client,
"/var/lib/images/app",
"registry.example.com/team/restored:release",
layout_tag: "release"
)
```
OCI artifact manifests retain their `subject`, `artifactType`, annotations,
and arbitrary payload descriptors:
```elixir
{:ok, artifact_metadata} =
Regc.Artifact.push(
client,
"registry.example.com/team/app:sbom",
[
{"application/spdx+json", {:stream, sbom_chunks}}
],
artifact_type: "application/vnd.example.sbom",
subject: "registry.example.com/team/app:release"
)
{:ok, artifact} =
Regc.Artifact.pull(
client,
"registry.example.com/team/app@#{artifact_metadata.digest}"
)
Enum.each(artifact.blobs, fn content ->
Enum.each(content.reader, &consume_chunk/1)
end)
Regc.Artifact.close(artifact)
```
Artifact pulls expose incrementally verified blob readers. Consume each reader
to EOF or call `Regc.Artifact.close/1`.
## Immutable image mutation
`Regc.Mutate` transforms manifest and image-config documents in memory. It
does not write to a registry or OCI layout. Input and output descriptors are
verified, unchanged documents retain their exact bytes, and every operation
returns a new mutation value plus an explicit change record.
```elixir
{:ok, image} =
Regc.Manifest.get(
client,
"registry.example.com/team/app:release"
)
{:ok, config_reader} =
Regc.Blob.get(
client,
"registry.example.com/team/app",
image.config
)
config_raw =
config_reader
|> Enum.to_list()
|> IO.iodata_to_binary()
with {:ok, mutation} <- Regc.Mutate.new(image, config_raw),
{:ok, mutation} <-
Regc.Mutate.Config.put_label(
mutation,
"org.example.release",
"stable"
),
{:ok, result} <- Regc.Mutate.finalize(mutation),
{:ok, _config} <-
Regc.Blob.put(
client,
"registry.example.com/team/app",
result.config_raw,
descriptor: result.config_descriptor
),
{:ok, _manifest} <-
Regc.Manifest.put(
client,
"registry.example.com/team/app:stable",
result.manifest_raw,
media_type: result.manifest_descriptor.media_type
) do
{:ok, result}
end
```
Manifest/index annotations and OCI/Docker media-type conversion live in
`Regc.Mutate.Manifest`. Labels, volumes, exposed ports, and reproducible
timestamp normalization live in `Regc.Mutate.Config`. Layer removal and exact
prefix rebasing live in `Regc.Mutate.Layer`. `Regc.Mutate.finalize/2` can also
materialize the documents with SHA-256, SHA-384, or SHA-512 descriptors.
## Check a tag
Platform selection is optional. This makes the default safe for checking a
list of tags whose platforms are not known in advance:
```elixir
case Regc.check_image("registry.example.com/team/app:1.2.3") do
{:ok, image} ->
Enum.map(image.platforms, fn platform ->
{platform.os, platform.architecture, platform.variant}
end)
{:error, error} ->
{error.stage, error.code, error.message}
end
```
For an image index, this request fetches and validates only the root manifest.
It does not choose a child based on the machine running Regc:
```elixir
image.platform # => nil
image.selected_ref # => nil
image.manifest # => nil
image.config_descriptor # => nil
image.layer_descriptors # => []
```
The root metadata remains available through `image.root_ref` and
`image.root_manifest`.
If the tag points directly to a single-image manifest, that manifest is
selected automatically. Its platform is not known until its configuration is
inspected.
## Select a platform
Pass an OCI platform string to resolve an index to an exact image manifest:
```elixir
{:ok, image} =
Regc.check_image("registry.example.com/team/app:1.2.3",
platform: "linux/amd64"
)
image.root_ref.digest
image.selected_ref.digest
image.config_descriptor.digest
Enum.map(image.layer_descriptors, & &1.digest)
```
Platforms have the form `os/architecture` or
`os/architecture/variant`, such as `linux/amd64` or `linux/arm/v7`.
When an index distinguishes images by additional OCI platform qualifiers,
pass a map so selection remains exact:
```elixir
Regc.check_image(reference,
platform: %{
os: "windows",
architecture: "amd64",
os_version: "10.0.20348.2402",
os_features: ["win32k"]
}
)
```
The map also accepts `:variant` and `:features`. CPU `:features` can be
verified only when selecting a descriptor from an image index; requesting
them for a direct image manifest returns `:platform_features_unverifiable`.
If multiple manifests match the base OS and architecture but differ by
qualifiers, Regc returns `:ambiguous_platform` instead of choosing one
arbitrarily.
Use `platform: :local` only when selecting the OS and architecture of the
runtime machine is intentional:
```elixir
Regc.check_image(reference, platform: :local)
```
## Inspect the image configuration
`inspect_image/2` fetches, verifies, and decodes the selected image
configuration:
```elixir
{:ok, image} =
Regc.inspect_image("registry.example.com/team/app:1.2.3",
platform: "linux/amd64"
)
image.platform
image.config.created
image.config.os
image.config.architecture
image.config.raw["config"]["Cmd"]
```
An explicit platform is required when the tag points to an image index.
For a direct single-image manifest, the platform can be omitted; Regc reads it
from the verified configuration.
If only the creation timestamp is needed:
```elixir
{:ok, created} =
Regc.image_created("registry.example.com/team/app:1.2.3",
platform: "linux/amd64"
)
```
## Platform behavior
| Operation | Direct image manifest | Image index |
| --- | --- | --- |
| `check_image/2` without a platform | Selects the manifest; platform remains unknown | Validates the root and reports `image.platforms` |
| `check_image/2` with a platform | Selects the manifest; platform remains unknown until inspection | Resolves the matching child |
| `inspect_image/2` without a platform | Inspects the config and infers the platform | Returns `:platform_required` |
| `inspect_image/2` with a platform | Inspects and validates the config platform | Resolves, inspects, and validates the matching child |
## Errors
Public functions return `{:ok, result}` or a structured error tuple. The
general registry primitives return `%Regc.Error{}`; the original image
inspection convenience functions retain `%Regc.Oci.Error{}`. Both include the
failing stage, a stable code, a human-readable message, and whether retrying
may succeed:
```elixir
case Regc.check_image(reference) do
{:ok, image} ->
image
{:error, error} ->
%{
stage: error.stage,
code: error.code,
message: error.message,
retryable?: error.retryable?
}
end
```
## References and transport
Docker-style shorthand references are expanded using Docker Hub defaults:
```text
redis:latest -> docker.io/library/redis:latest
bitnami/redis:7 -> docker.io/bitnami/redis:7
```
References must include a tag or a SHA-256, SHA-384, or SHA-512 digest.
Registry-qualified references remain supported:
```text
registry.example.com/team/app:1.2.3
registry.example.com/team/app@sha256:0123456789abcdef...
```
HTTPS is the default. Plain HTTP is accepted automatically for local
development registries:
```elixir
Regc.check_image("http://localhost:5000/team/app:dev")
```
For any other plaintext registry, both the `http://` scheme and
`allow_insecure: true` are required unless that host has `tls: :disabled`.
Redirects are followed within a bounded request chain. HTTPS targets are
accepted; plaintext targets still require `allow_insecure: true`, a host with
`tls: :disabled`, or two local development endpoints. Authorization, proxy
authorization, and cookie headers are removed when the redirect changes
origin. Redirects from public hosts to targets that resolve to non-global
addresses, including private, loopback, link-local, and special-use ranges,
are rejected, and the validated target addresses are pinned to the redirected
connection.
## Options
| Option | Default | Purpose |
| --- | --- | --- |
| `:platform` | omitted | Select `os/architecture[/variant]`, a qualifier map, or `:local` |
| `:max_manifest_bytes` | 4 MiB | Limit each downloaded manifest |
| `:max_config_bytes` | 4 MiB | Limit the downloaded image configuration |
| `:max_index_depth` | `8` | Limit nested index traversal during platform resolution |
| `:max_index_manifests` | `64` | Limit child manifests fetched while resolving nested indexes |
| `:max_redirects` | `5` | Limit HTTP redirects followed for one metadata request |
| `:max_retries` | `2` | Retry replay-safe requests after transient failures |
| `:retry_backoff` | 100 ms | Set the initial exponential retry delay |
| `:retry_max_delay` | 5 seconds | Cap retry and `Retry-After` delays |
| `:connect_timeout` | 5 seconds | Limit connection establishment time |
| `:timeout` | 15 seconds | Limit total HTTP response time, including redirects |
| `:allow_insecure` | `false` | Permit explicit plain HTTP for a non-local registry |
| `:resolver` | built-in DNS | Use an arity-2 DNS resolver function |
| `:transport` | built-in HTTP | Use an arity-1 request function or a module implementing `Regc.Oci.Transport` |
| `:transport_options` | `[]` | Pass a keyword list to a transport module through `Regc.Oci.Options` |
### Custom transports
An arity-1 transport function receives a `%Regc.Oci.Transport.Request{}` and
returns `{:ok, response}` or `{:error, reason}`. A successful response can be a
`%Regc.Oci.Transport.Response{}` or a map with `:status`, `:headers`, and
`:body` fields.
For a reusable transport with configuration, implement the
`Regc.Oci.Transport` behaviour:
```elixir
defmodule MyTransport do
@behaviour Regc.Oci.Transport
alias Regc.Oci.{Options, Transport}
alias Regc.Oci.Transport.{Request, Response}
@impl Transport
def request(%Request{} = request, %Options{} = options) do
client = Keyword.fetch!(options.transport_options, :client)
# Perform the requested method and body exchange with client.
{:ok, %Response{status: 200, headers: %{}, body: "..."}}
end
end
Regc.check_image(reference,
transport: MyTransport,
transport_options: [client: client]
)
```
The resolver function receives `(host, family)`, where `family` is `:inet` or
`:inet6`, and returns `{:ok, addresses}` or `{:error, reason}`. Custom
transports that follow redirects or resolve hosts independently are responsible
for equivalent timeout, redirect, TLS, and network-target protections.
## Verification
Regc verifies:
- requested, registry-provided, and parent-descriptor manifest digests;
- parent-declared manifest sizes;
- image-config digest and size;
- media types and required manifest/config structure;
- exact platform selection and, during inspection, config-platform agreement when requested; and
- configured response-size and nested-index limits.
## Current scope
Supported:
- anonymous, Basic, Bearer, refresh-token, and direct-token registry access;
- reusable clients with token caching, connection pooling, host aliases,
mirrors, custom TLS, retries, and per-host request limits;
- manifest `GET`/`HEAD`/`PUT`/`DELETE`;
- blob `GET`/`HEAD`/`PUT`/mount/`DELETE`, including streamed verified reads,
monolithic uploads, and resumable chunked upload sessions;
- native and compatibility-fallback tag deletion;
- paginated tags, repository catalog, and OCI referrers with fallback;
- OCI image indexes, image manifests, and artifact manifests;
- Docker manifest lists and schema-2 image manifests;
- nested image indexes;
- deduplicated blob and recursive image graph copy with optional platform,
referrer, and digest-tag handling;
- manifest-only retagging;
- local OCI image-layout storage and registry/layout import and export;
- streamed OCI artifact push, pull, and copy with subject relationships;
- immutable manifest, config, timestamp, layer, rebase, media-type, and digest
mutation APIs;
- SHA-256, SHA-384, and SHA-512 content verification; and
- verified image-config retrieval, including inline config data.
Not currently supported:
- Docker schema-1 manifests or Docker archive import/export.
## Development
The project targets Elixir `~> 1.20` and has no runtime package dependencies.
```shell
mix test
mix format --check-formatted
mix compile --warnings-as-errors
```