inference

# Inference

<p align="center">
  <img src="assets/inference.svg" alt="Inference" width="200" />
</p>

<p align="center">
  <a href="https://opensource.org/licenses/MIT">
    <img alt="MIT License" src="https://img.shields.io/badge/license-MIT-0f172a?style=for-the-badge" />
  </a>
  <a href="https://github.com/nshkrdotcom/inference">
    <img alt="GitHub" src="https://img.shields.io/badge/github-nshkrdotcom%2Finference-111827?style=for-the-badge&logo=github" />
  </a>
</p>

`inference` provides reusable Elixir contracts for semantic model inference:
requests, responses, clients, adapters, capabilities, trace metadata,
redaction, and adapter conformance tests.

It is intentionally small. The package gives application code one stable shape
for prompts, responses, client configuration, trace summaries, and adapter
contracts. Provider SDKs, local agent runtimes, governed execution systems, and
transport stacks remain outside the core contract.

It is not an Execution Plane wrapper. Execution Plane remains the lower runtime
substrate. `inference` is the product-facing provider/model boundary used by
standalone libraries such as `trinity_coordinator` and `gepa_ex`.

## Installation

Add `:inference` to the application that wants the shared contract:

```elixir
def deps do
  [
    {:inference, "~> 0.2.0"}
  ]
end
```

Provider-specific dependencies are opt-in. For example:

```elixir
def deps do
  [
    {:inference, "~> 0.2.0"},
    {:gemini_ex, "..."},
    {:agent_session_manager, "..."}
  ]
end
```

The initial package ships adapter modules, not separate adapter packages:

- `Inference.Adapters.Mock`
- `Inference.Adapters.ASM`
- `Inference.Adapters.GeminiEx`
- `Inference.Adapters.ReqLlmNext`
- `Inference.Adapters.ReqLLM`

Provider-specific dependencies are opt-in dependencies in the consuming
application. `GeminiEx` is the direct Gemini API adapter: Gemini API users add
both `:inference` and `:gemini_ex`; core/mock users add only `:inference`.
Gemini CLI is retired. Antigravity is the current Google coding-agent SDK and
is reached through the explicitly admitted ASM agent-session boundary, not the
Gemini API adapter.

Jido governed execution is owned by `jido_integration`. The Jido-owned adapter
implements `Inference.Adapter` from that repository and translates shared
requests into governed control-plane execution.

That governed lane is required release scope for platforms that use universal
auth authority. It remains outside the core `:inference` package so direct
standalone adapters stay reusable, while governed deployments carry authority
refs, credential handles or leases, target grants, and redacted trace evidence
through the Jido-owned adapter.

## Usage

```elixir
client =
  Inference.Client.new!(
    adapter: Inference.Adapters.Mock,
    provider: :mock,
    model: "mock-fast",
    adapter_opts: [response_text: "hello"]
  )

{:ok, response} = Inference.complete(client, "Say hello")
Inference.Response.text(response)
```

Requests can also be built explicitly:

```elixir
{:ok, request} =
  Inference.Request.from_messages([
    %{role: :system, content: "Be concise."},
    %{role: :user, content: "Summarize the result."}
  ])

{:ok, response} = Inference.complete(client, request)
```

## Design Rules

- The default test path is deterministic and mock-only.
- Live provider calls are examples, not test requirements.
- Provider dependencies are installed by the consuming application.
- Adapter modules translate to and from provider libraries; they do not hide
  provider setup, credentials, or runtime requirements.
- Clients admit model and local-model endpoints by default. Agent-session
  adapters require an explicit opt-in — `Inference.Client.agent_session!/1`, or
  `admitted_kinds: [:agent_session]` — so a generic inference caller cannot
  silently flatten a stateful coding-agent session.
- `Inference.Request.response_format` is a closed union: `nil`, `:text`,
  `{:json, :object}`, or
  `{:json_schema, %{name: name, schema: schema, strict: strict?}}`. Every
  adapter maps the declared format onto a real provider option or refuses it
  with a `:response_format_unsupported` error. Dropping a declared format is a
  contract violation.
- Adapters report `Inference.Capability` claims through
  `Inference.capabilities/1`, so a registry can ask "does this provider accept
  JSON Schema?" before dispatch. An adapter that cannot establish a claim
  reports `:unknown`; it never assumes support.
- `Inference.Adapters.ASM` is common-only and completion-only. ASM owns its own
  option contract (`ASM.Options.validate/2`), so the adapter does not re-impose
  ASM's strict-common preflight from the outside; it maps
  `{:json_schema, _}` onto ASM's `:output_schema`, locks the completion-only
  provider profile, and rejects provider-native tool/configuration keys until
  ASM has a proven all-provider tool contract.
- Provider errors keep their cause: `ASM.Error` kinds and `Gemini.Error`
  http statuses map onto the declared `:timeout`, `:rate_limited`,
  `:missing_credentials`, `:missing_dependency`, `:invalid`, and
  `:invalid_response` categories, and the provider's own error value is
  preserved under `metadata.provider_error`.
- Jido governed execution is owned by `jido_integration`, which implements
  `Inference.Adapter` from the Jido side.
- Direct `:inference` adapters are standalone mechanics. They do not decide
  durable provider credential authority, target attachment, or workflow
  admission for governed execution.
- Shared governed clients may carry explicit authority refs, endpoint refs,
  provider-account refs, credential refs, service-identity refs, target refs,
  and redaction values. The shared package rejects direct provider keys,
  endpoint auth, model-account secrets, service identity secrets, raw env
  functions, and adapter defaults beside that authority packet; Jido still owns
  the governed adapter and durable control-plane behavior.

## Guides

- [Architecture](architecture.html)
- [Requests and Responses](requests-and-responses.html)
- [Clients and Adapters](clients-and-adapters.html)
- [Optional Providers](optional-providers.html)
- [Adapter Testkit](adapter-testkit.html)
- [Live Examples](live-examples.html)
- [Jido Integration Ownership](jido-integration.html)