Packages

A library for native agent/server communication over TLS

Current section

Files

Jump to
node_wire README.md
Raw

README.md

# NodeWire

[![Hex Version](https://img.shields.io/hexpm/v/node_wire.svg)](https://hex.pm/packages/node_wire)
[![Hex Docs](https://img.shields.io/badge/docs-HexDocs-blue.svg)](https://hexdocs.pm/node_wire/)

**A direct line between your agents and your server.**

NodeWire gives Elixir agents a persistent, authenticated connection to their
server. You define the messages, authenticate the peers and decide when each
connection is ready for work. NodeWire handles TLS, protocol negotiation,
heartbeats, bounded send queues and reconnects, with `:telemetry` events to show
you what's happening along the way.

Both endpoints use locally compiled `NodeWire.Protocol` modules. Your application
supplies the authentication, message validation and handlers.

See the [changelog](CHANGELOG.md) for release history and compatibility policy.

## Requirements

- Elixir 1.18 or later.
- Erlang/OTP 27 or later. NodeWire uses OTP's built-in `:json` module for
  handshake and control messages, including when application messages use ETF.

## Installation

The package can be installed by adding `node_wire` to your list of dependencies in `mix.exs`:

```elixir
def deps do
  [
    {:node_wire, "~> 0.1.0"}
  ]
end
```

Run `mix deps.get` after adding the dependency.

## Quickstart

### (Optional)1. Create a local development certificate

From your application's root, use OpenSSL to create a local CA and a signed
certificate for `localhost`. These files are only for the local example; keep
`tmp/` and its private keys out of version control.

```sh
mkdir -p tmp/nodewire
openssl req -x509 -newkey rsa:2048 -nodes -sha256 -days 30 \
  -subj "/CN=NodeWire local CA" \
  -addext "basicConstraints=critical,CA:TRUE" \
  -addext "keyUsage=critical,keyCertSign,cRLSign" \
  -keyout tmp/nodewire/ca.key -out tmp/nodewire/ca.pem

openssl req -new -newkey rsa:2048 -nodes -sha256 \
  -subj "/CN=localhost" \
  -keyout tmp/nodewire/server.key -out tmp/nodewire/server.csr

cat > tmp/nodewire/server.ext <<'EOF'
basicConstraints=critical,CA:FALSE
keyUsage=critical,digitalSignature,keyEncipherment
extendedKeyUsage=serverAuth
subjectAltName=DNS:localhost
EOF

openssl x509 -req -in tmp/nodewire/server.csr \
  -CA tmp/nodewire/ca.pem -CAkey tmp/nodewire/ca.key -CAcreateserial \
  -sha256 -days 30 -extfile tmp/nodewire/server.ext \
  -out tmp/nodewire/server.pem
```

The client below trusts `tmp/nodewire/ca.pem`. For deployment, use a
persistent certificate issued by your CA, its matching hostname and your own
credentials.

### 2. Declare a protocol and handlers

Place these modules under your application's `lib/` directory:

```elixir
defmodule MyApp.Wire do
  use NodeWire.Protocol,
    id: "myapp.agent",
    version: 1,
    codec: :erlang_term,
    capabilities: ["echo.v1"]

  message(:echo, id: 1)
end
```

Declare separate server and client handlers. This small example requires no
application initialization, so its initial phase is already ready:

```elixir
defmodule MyApp.ServerHandler do
  use NodeWire.Handler
  def init(expected_token), do: {:ok, expected_token}
  # Replace this example with your application's constant-time credential check.
  def authenticate(%{"node" => node, "token" => token}, _conn, token),
    do: {:ok, node, nil}

  def authenticate(_, _, state), do: {:error, :denied, state}
  def validate_message(:echo, p, _, _) when is_binary(p), do: {:ok, p}
  def validate_message(_, _, _, _), do: {:error, :invalid_echo}

  def handle_message({:echo, p}, _, state),
    do: {:noreply, state, [{:send, {:echo, p}}]}
end

defmodule MyApp.ClientHandler do
  use NodeWire.Handler
  def init(credentials), do: {:ok, credentials}
  def handshake_data(_, credentials), do: {:ok, credentials, nil}
  def validate_message(:echo, p, _, _) when is_binary(p), do: {:ok, p}
  def validate_message(_, _, _, _), do: {:error, :invalid_echo}

  def handle_message({:echo, p}, _, state) do
    IO.puts(p)
    {:noreply, state}
  end
end
```

### 3. Start both endpoints

In `MyApp.Application.start/2`, start these children under your application's
supervisor. Both endpoints begin in `:ready` because this echo example has no
additional initialization phase:

```elixir
phases = [
  initial: :ready,
  ready: :ready,
  states: %{ready: %{incoming: [:echo], outgoing: [:echo], next: []}}
]

server =
  {NodeWire.Server,
   name: MyApp.AgentServer,
   protocols: [MyApp.Wire],
   handler: MyApp.ServerHandler,
   handler_args: "example-token",
   phases: phases,
   transport:
     {:tls,
      [
        port: 5000,
        options: [certfile: ~c"tmp/nodewire/server.pem", keyfile: ~c"tmp/nodewire/server.key"]
      ]}}

client =
  {NodeWire.Client,
   name: MyApp.AgentClient,
   protocols: [MyApp.Wire],
   handler: MyApp.ClientHandler,
   handler_args: %{"node" => "agent-01", "token" => "example-token"},
   phases: phases,
   transport:
     {:tls, [host: "localhost", port: 5000, options: [cacertfile: ~c"tmp/nodewire/ca.pem"]]}}

Supervisor.start_link([server, client], strategy: :one_for_one, name: MyApp.Supervisor)
```

Start the application with `iex -S mix`. Once the client has negotiated, send an
echo; the client handler prints `hello`:

```elixir
NodeWire.Client.status(MyApp.AgentClient)
# Wait until :connection contains a %NodeWire.Conn{phase: :ready} snapshot.
NodeWire.Client.send(MyApp.AgentClient, {:echo, "hello"})
# => :ok
```

Starting the supervisor does not wait for negotiation or application readiness.
For application code, use lifecycle observation or reconcile connection status
before sending. An early send returns `{:error, :not_connected}` or
`{:error, :not_ready}`.

## TLS configuration

Client TLS defaults to `verify: :verify_peer`. With verification enabled, the
certificate must authenticate `host`, and the client supplies SNI and HTTPS
hostname matching. Configure trusted CAs through the TLS options.

Applications that authenticate the server path through a trusted network or
VPN can explicitly accept any server certificate without a CA file:

```elixir
[transport: {:tls, [host: "agents.internal", port: 5000, options: [verify: :verify_none]]}]
```

With `verify_none`, TLS encrypts traffic but does not authenticate the server.
The client sends its handshake credentials after TLS connects. The application
owns this trust decision, including any server-supplied code it executes.
The test certificates and private key under `test/fixtures/` are public fixtures
and must never be deployed.

## Protocol and validation

Protocol definitions are immutable: change version for incompatible changes.
Names and IDs must be unique; versions and IDs are positive uint16 values.
Every configured version belongs to one protocol family. Negotiation selects the
highest exact version/codec match satisfying both peers' required capabilities.
Use `handshake: [required_capabilities: ["echo.v1"], timeout_ms: 10_000]`.

Built-in codecs are `:erlang_term` (`erlang-term.v1`) and `:json` (`json.v1`).
ETF uses safe decoding and rejects compressed ETF and trailing bytes. Preload
allowlisted atoms used by your domain. Safe ETF does not validate term semantics;
provide bounded domain validators for every inbound message. JSON rejects
invalid UTF-8, duplicate keys (including escaped equivalents), trailing input,
depth over eight, and collections over 1,024 entries during parsing.

Inner frames are `<<"NW", 1, kind, code::16, 0::16, payload::binary>>`; TLS
`packet: 4` supplies the separate length prefix. HEARTBEAT is exactly
`4E57010100040000`. HELLO/ACCEPT/REJECT/CLOSE are strict, bounded control JSON,
independent of the application codec. No controls reach application handlers.

## Callbacks and actions

`use NodeWire.Handler` declares the behaviour. Implement `init/1`,
`handle_message/3` and inbound validation; servers also implement
`authenticate/3`, clients `handshake_data/2`. All ordinary callbacks run in one
connection owner. Keep them short. Authentication is constrained by the bounded
handshake worker and native deadline; timeout kills that attempt.

| Callback | Return |
| --- | --- |
| `init(args)` | `{:ok, state}` or `{:stop, reason}` |
| `authenticate(credentials, conn, state)` | `{:ok, identity, state}` or `{:error, reason, state}` |
| `handshake_data(conn, state)` | `{:ok, credentials, state}` or `{:error, reason, state}` |
| `validate_message(name, payload, conn, state)` | `{:ok, validated_payload}` or `{:error, reason}` |
| `validate_outgoing(name, payload, conn, state)` | `{:ok, payload}` or `{:error, reason}` |
| `handle_negotiated(conn, state)` | normal callback result |
| `handle_message({name, payload}, conn, state)` | normal callback result |
| `handle_info(message, conn, state)` | normal callback result |
| `handle_sent(tag, conn, state)` | `{:noreply, state}` or `{:stop, reason, state}` |
| `handle_phase_change(old, new, conn, state)` | `{:noreply, state}` or `{:stop, reason, state}` |
| `handle_disconnect(reason, conn, state)` | `:ok`, best effort |

Normal results are `{:noreply, state}`, `{:noreply, state, actions}` or
`{:stop, reason, state}`. Defaults: inbound validation fails closed, outgoing
validation passes through, optional lifecycle callbacks do nothing. Role-specific
callbacks are checked at startup.

Actions execute in order:

```elixir
{:noreply, state,
 [
   {:send, {:ready, ready_payload}},
   {:phase, :ready}
 ]}
```

Other actions are `{:send, message, tag: tag}` and terminal `{:close, reason}`.
A successful tagged write commits `handle_sent/3` before the next incoming frame
or public-send response. Hooks cannot return actions. Action shapes, IDs and
phase paths are checked before execution; I/O failures stop later actions.
Earlier writes cannot be rolled back. Callback failures close the connection.
Never synchronously send to your own connection from a callback.

`conn` is a `%NodeWire.Conn{}` snapshot with identity, unique instance ID, role,
selected protocol, version, capabilities, phase, peer and opaque owner routing
and admission handles.
It contains no credentials or socket. Use `NodeWire.Connection.notify(conn, msg)`
from asynchronous work; `:ok` means queued for that exact live instance.
Old references return `{:error, :stale_connection}`. Notifications accept
`timeout_ms:` as an optional third argument. A caller timeout returns
`{:error, {:outcome_unknown, :timeout}}`; the queued notification may still be
processed, so do not replay it automatically. Ordinary application mailbox
messages, including `Process.send_after(self(), ...)`, reach `handle_info/3`;
socket events and internal timers are intercepted.

## Phases, deadlines and limits

Configure `phases: [initial: :initializing, ready: :ready, states: %{...}]`.
Each state has exact `incoming`, `outgoing`, and `next` lists. All messages must
exist in every configured version, targets must exist and ready must be reachable.
Inbound allowlists are checked before decoding. Phase changes publish only after
the phase callback succeeds. ACCEPT means negotiated; application readiness is
an explicit phase action.

`readiness_timeout_ms` defaults to 35,000 and starts after ACCEPT. Only reaching
ready cancels it. Heartbeats never extend it. Client heartbeat send defaults to
10,000 ms; server receive timeout defaults to 30,000 ms. Override via
`heartbeat: [send_interval_ms: nil, receive_timeout_ms: nil]` when appropriate.
Only valid inbound heartbeat controls refresh the receive timeout.

Transport/native defaults: TLS handshake 5,000 ms, native handshake 10,000 ms,
application packet 16,777,216 bytes, handshake/control packet 16,384 bytes,
1,024 connections and 128 simultaneous handshakes. Override frame and concurrency
caps with `limits: [...]`, and TLS deadline with `tls_handshake_timeout_ms`.

Public sends and notifications share a per-session admission limit of
256 pending messages and 16,777,216 bytes. Configure these with
`limits: [max_pending_messages: ..., max_pending_bytes: ...]`. Admission counts
the Erlang external representation of the request, including send options and
tracking tags. The encoded packet limit still applies independently.
An overloaded connection returns `{:error, :busy}` before the payload enters
the session mailbox. That rejection is safe to retry on the same connection.
Admission remains responsive while the session is busy running a callback.

A reservation is released after the session processes the request. Notifications
retain their reservation through their application callback, including after the
caller receives `:ok`. Producer death cancels work that the session has not
accepted yet. Accepted work keeps its reservation until completion. Session death
removes all its reservations. Applications own any deferred queue or sample
coalescing policy.

Control schemas additionally hard-cap offers at eight, capabilities at 32,
credentials at 16 string fields and 4,096 encoded bytes (64-byte keys,
1,024-byte values). Packet caps are installed in SSL before receiving frames.
Configuring more than eight protocol versions raises `ArgumentError` at startup.

## Public APIs and delivery meaning

```elixir
NodeWire.Server.send(server, identity, {name, payload}, on_sent: tag, timeout_ms: 5000)
# {:ok, conn} | :error
NodeWire.Server.connection(server, identity)
# initializing and ready snapshots
NodeWire.Server.connections(server)
NodeWire.Server.disconnect(server, identity, :normal)
# useful with port: 0
NodeWire.Server.port(server)
NodeWire.Client.send(client, {name, payload}, timeout_ms: 5000)
# state, connection snapshot, reason
NodeWire.Client.status(client)
# disables retries
NodeWire.Client.disconnect(client)
# explicit fresh attempt
NodeWire.Client.reconnect(client)
```

`:ok` means accepted by local SSL and post-send tracking committed. It does not
prove remote delivery or execution. No application messages are automatically
retried or replayed. Errors before writing include `:not_connected`, `:not_ready`,
`{:message_not_allowed, phase}` and `{:invalid_outbound, reason}`.
`{:outcome_unknown, reason}` covers write errors, post-write hook failure, and
caller timeout after dispatch. Treat these as potentially executed operations.

The client survives session loss, resets handler state and uses generation-tagged
fixed-delay reconnect timers (default 20,000 ms). Authentication, duplicate identity,
incompatible protocol and TLS verification failures stop automatic retry.
Network failures and readiness/heartbeat timeouts retry. Explicit application
rejection defaults to permanent. `reconnect_policy` is a bounded map of failure
atoms to `:transient` / `:permanent`; it cannot override authentication or trust
failures. `reconnect: [enabled: false]` disables automatic attempts.

The server's monitored registry claims identity atomically and cleans up after
forced kills. Registry loss restarts the coherent connection tree. An optional
`lifecycle_observer: pid_or_registered_name` receives
`{:nodewire, :connected | :ready | :disconnected, conn, reason}`. These events are
best effort; reconcile using `connections/1`. Debug Logger events contain safe
connection metadata, never raw HELLOs or payloads. `send_accepted` is local only.

## Failure reporting and telemetry

Successful connection attempts clear earlier failure reasons. Unexpected owner
death is treated as a fresh transport loss, so an old authentication rejection
cannot disable recovery after a successful explicit reconnect.

The lifecycle observer receives the actual local disconnect cause when the
session closes normally. Abrupt owner death falls back to `:transport_closed`.
Remote CLOSE reasons are retained from the fixed control-protocol allowlist.
Inbound codec, phase, validation, and callback failures use
`%NodeWire.Error{stage: ..., code: ..., connection_id: ...}`. Outbound validation
and packet errors return `{:error, {:invalid_outbound, error}}`. Callback
exceptions expose the callback name without its exception message or arguments.
Structured callback rejection details are replaced with `:application_rejected`.

Telemetry event names use the `[:node_wire, event]` prefix. Lifecycle events
continue to produce debug logs. Performance events include:

| Event | Measurements | Additional metadata |
| --- | --- | --- |
| `:admission_accepted` | `pending_messages`, `pending_bytes` |  |
| `:admission_rejected` | `pending_messages`, `pending_bytes`, `bytes` | `reason: :busy` |
| `:queue_wait` | `duration`, `queue_depth` |  |
| `:callback` | `duration`, `queue_depth` | `callback`, application `message_type` where available |
| `:socket_write` | `duration`, `queue_depth`, `bytes` | `message_type` |
| `:message_received` | `bytes` | `message_type` |

Durations use monotonic native time units. Queue wait starts after admission
and ends when the session begins processing the request or notification
callback. Callback execution time is reported separately. Metadata includes
connection ID, endpoint, role, phase, protocol ID and version. Payloads,
credentials, callback arguments and exception text are excluded. Keep telemetry
handlers fast because they execute in the process emitting the event.

## Development

Application-specific adapters, runtime installers, credentials policy and job
correlation belong in the consuming application.

Fetch dependencies with `mix deps.get`, then run:

```sh
mix format --check-formatted
mix compile --warnings-as-errors
mix test
MIX_ENV=prod mix compile --warnings-as-errors
mix docs --warnings-as-errors
```

Generated documentation is written to `doc/`. Tests use real verified TLS,
including the separate OTP packet boundary, malformed contracts,
phase-before-codec tracing, duplicate identity, hung authentication, lifecycle recovery, concurrency,
heartbeats, reconnect generations and ambiguous post-write failure.

## License

NodeWire is released under the MIT License. See [LICENSE](LICENSE).