system_one_client

# SystemOne Client

Elixir client for System One decision models: [TypeSafe Jev](https://docs.typesafe.ai),
[Cloudflare Clef](https://developers.cloudflare.com/workers-ai/models/clef/),
[OpenAI's Decisions API](https://developers.openai.com/api/docs/guides/decisions),
[OpenRouter's decisions endpoint](https://openrouter.ai), and anything self-hosted that
speaks the `/v1/systemone` wire format (for example [CLM](https://github.com/Contrastive-LM/CLM)).

You send a `state` (any JSON) and a map of typed questions; you get back typed answers with
their **full probability distributions**, never collapsed to a single pick. That is what a
router or a gate needs: top-k over a Choice, a threshold on a Noul, a level cut on a Score.

- One behaviour, `SystemOneClient`, with `evaluate(state, questions, opts)`.
- `SystemOneClient.HTTP` for real calls; `SystemOneClient.Stub` for tests, no network.
- Five providers behind one option; `url:` and `model:` override any of them.
- Retries on 429, 529, 502, 503, 504 and transport errors, honouring `Retry-After`.
- Never raises for network conditions; tagged error tuples at the boundary.
- Telemetry without leaking state, questions, or keys.
- Depends on `req`, `jason`, `telemetry` only. Apache 2.0.

## Install

```elixir
def deps do
  [{:system_one_client, "~> 0.1"}]
end
```

## Use

```elixir
questions = %{
  "needs_tool" => %{
    "type" => "noul",
    "instructions" => "Does the next step need one of the actions in `available_actions`?"
  },
  "tool" => %{
    "type" => "choice",
    "instructions" => "Which action should run next to make progress on `request`?",
    "criteria" => %{"add" => "Adds two numbers", "gcd" => "Greatest common divisor", "none" => "No action fits"}
  },
  "depth" => %{
    "type" => "score",
    "instructions" => "How much reasoning does `request` need?",
    "criteria" => ["One direct step", "A few dependent steps", "A long chain or a known trap"]
  }
}

state = %{request: "what is the gcd of 1071 and 462", available_actions: ["add", "gcd"]}

{:ok, answers, meta} = SystemOneClient.evaluate(state, questions)

answers["tool"]
#=> %SystemOneClient.Answer.Choice{choice: "gcd", confidence: 0.97,
#     probabilities: %{"gcd" => 0.97, "add" => 0.02, "none" => 0.01}}
answers["needs_tool"]   #=> %SystemOneClient.Answer.Noul{noul: 0.96}
answers["depth"]        #=> %SystemOneClient.Answer.Score{score: 0.1, probabilities: %{...}, confidence: 0.9, legend: ...}
meta                    #=> %{model: "jev-1.13.0", usage: %{...}, latency_ms: 134, provider: :typesafe}
```

Question maps are the API's wire format verbatim; this library adds no schema of its own.
See TypeSafe's [primitives](https://docs.typesafe.ai/primitives) for `noul`, `choice`
and `score`.

## Providers

| `provider:` | Endpoint | Key | Default model |
| --- | --- | --- | --- |
| `:typesafe` (default) | `https://api.typesafe.ai/v1/systemone` | `api_key:` or `TYPESAFE_API_KEY` | `jev-latest` |
| `:cloudflare` | `https://api.cloudflare.com/client/v4/accounts/{account_id}/ai/run/@cf/cloudflare/clef` | `api_key:` or `CLOUDFLARE_API_TOKEN`; `account_id:` or `CLOUDFLARE_ACCOUNT_ID` | `clef` (`model: "clef-flash"` for the 9B) |
| `:openai` | `https://api.openai.com/v1/decisions` | `api_key:` or `OPENAI_API_KEY` | `gpt-6-luna` |
| `:openrouter` | `https://openrouter.ai/api/alpha/decisions` | `api_key:` or `OPENROUTER_API_KEY` | `typesafe/jev-1.13` |
| `:compatible` | `url:` (required) | optional | none |

```elixir
SystemOneClient.evaluate(state, questions, provider: :cloudflare, model: "clef-flash")
SystemOneClient.evaluate(state, questions, provider: :openai)
SystemOneClient.evaluate(state, questions, provider: :compatible, url: "http://localhost:8700/v1/systemone")
```

Cloudflare's `result` envelope is unwrapped for you; a failed envelope returns
`{:error, {:provider_error, errors}}`.

### OpenAI Decisions

OpenAI's Decisions API asks the same three kinds of question with a different wire
format, so `provider: :openai` translates in both directions and you keep writing the
same questions and get the same structs back:

- `noul` is sent as a `predicate`. Predicates have no criteria field, so `true` / `false`
  criteria are appended to the instructions as "True when: ..." and "False when: ...".
- `choice` criteria become `choices`, and `score` levels become `levels` labelled
  `"Level 0"`, `"Level 1"`, ... with your text as the description.
- `state` becomes the text `input`: strings pass through, anything else is JSON-encoded.
  Non-string instructions and descriptions are JSON-encoded too.
- Score probabilities are keyed by level index (`"0"`, `"1"`, ...) and `legend` carries
  your level text, the same as TypeSafe.
- OpenAI can refuse a single question. That answer comes back as
  `%SystemOneClient.Answer.Refusal{}` under its id, and the other answers are unaffected.
- A `choice` or `score` question without criteria is rejected before the request, as
  `{:error, {:invalid_question, id, :missing_criteria}}`.
- For regional processing, pass `url: "https://us.api.openai.com/v1/decisions"` (or `eu.`).

## Images

OpenAI's Decisions API and Cloudflare's Clef can judge images as well as text. Pass them
as base64 data URLs in the `images:` option; `SystemOneClient.image_data_url/2` builds one
from bytes:

```elixir
photo = SystemOneClient.image_data_url(File.read!("product.png"), "image/png")

SystemOneClient.evaluate(
  "Customer says the item arrived broken.",
  %{"visible_damage" => %{"type" => "noul", "instructions" => "Does the product show visible damage?"}},
  provider: :openai,
  images: [photo]
)
```

Both providers see the images first and then the state. For OpenAI the client builds a
user message with `input_image` parts followed by the state as text; for Clef it sends the
top-level `images` array.

Images are checked before any request:

- Every entry must be a `data:image/...;base64,` URL. Remote `http(s)` URLs are rejected,
  since neither API accepts them.
- Clef allows at most 4 PNG, JPEG or WebP images, 4 MiB each and 8 MiB in total once decoded.
- Providers without image support (`:typesafe`, `:openrouter`, `:compatible`) return
  `{:error, {:unsupported_option, :images, provider}}` rather than dropping them.

## Options

| Option | Default | Meaning |
| --- | --- | --- |
| `provider` | `:typesafe` | see above |
| `api_key`, `account_id`, `url`, `model` | provider defaults | explicit values always win |
| `receive_timeout` | `2_000` ms | per attempt |
| `max_retries` | `2` | on 429, 529, 502, 503, 504 and transport errors |
| `retry_delay_ms` | `250` | base of the exponential backoff; `Retry-After` / `retry-after-ms` win when present |
| `client` | app env `:client`, else `HTTP` | implementation module (use `SystemOneClient.Stub` in tests) |
| `plug` | none | Req plug, for `Req.Test` in your own tests |
| `env` | `System.get_env()` | replace the environment (tests) |
| `images` | none | base64 image data URLs (`:openai`, `:cloudflare`); see [Images](#images) |

Errors are tagged tuples:

```elixir
{:error, :missing_api_key | :missing_account_id | :missing_url | {:unknown_provider, p}}
{:error, {:invalid_question, id, reason}}
{:error, {:http, status, body} | {:transport, reason}}
{:error, {:malformed_answers, reason} | {:provider_error, errors}}
{:error, {:unsupported_option, :images, provider} | {:invalid_image, index, reason}}
{:error, {:too_many_images, count, max} | {:images_too_large, bytes, max}}
```

## Testing your code

```elixir
# config/test.exs
config :system_one_client, client: SystemOneClient.Stub

# in a test
alias SystemOneClient.Answer.{Choice, Noul}

{:ok, answers, _} =
  SystemOneClient.evaluate(state, questions,
    answers: %{
      "needs_tool" => %Noul{noul: 0.9},
      "tool" => %Choice{choice: "gcd", probabilities: %{"gcd" => 0.9, "add" => 0.1}, confidence: 0.9}
    })
```

`answers:` takes a map (structs or raw API maps) or a `fn state, questions -> map | {:error, _} end`.
For HTTP-level tests pass `plug: {Req.Test, MyTest}` and stub with `Req.Test.stub/2`.

## Telemetry

`[:system_one_client, :request, :start | :stop | :exception]` via `:telemetry.span/3`.
Metadata: `provider`, `model`, `question_count`, `image_count`; on `:stop` also `status` (`:ok` | `:error`),
`latency_ms`, and `error` (a kind such as `{:http, 429}` or `:transport`). State, question
text, bodies and keys are never included.

## Origin

Extracted from an experiment that used Jev as a tool and model router inside a
[Jido](https://github.com/agentjido/jido) AI agent; the experiment, its benchmarks and the
transformer that consumes this client live on
[schainks/jido, branch `experiment/jev-tool-routing`](https://github.com/schainks/jido/tree/experiment/jev-tool-routing).

## Running the tests

With Elixir installed: `mix test`. Without: `scripts/run.sh test` runs them in the
`hexpm/elixir:1.20.4` Docker image.