Packages
nous
0.15.8
0.17.0
0.16.6
0.16.5
0.16.4
0.16.3
0.16.2
0.16.1
0.16.0
0.15.8
0.15.7
0.15.6
0.15.5
0.15.4
0.15.3
0.15.2
0.15.1
0.15.0
0.14.3
0.14.2
0.14.1
0.14.0
0.13.3
0.13.2
0.13.1
0.13.0
0.12.17
0.12.16
0.12.15
0.12.14
0.12.13
0.12.12
0.12.11
0.12.9
0.12.7
0.12.6
0.12.5
0.12.3
0.12.2
0.12.0
0.11.3
0.11.0
0.10.1
0.10.0
0.9.0
0.8.1
0.8.0
0.7.2
0.7.1
0.7.0
0.5.0
AI agent framework for Elixir with multi-provider LLM support
Current section
Files
Jump to
Current section
Files
lib/nous/http/stream_backend.ex
defmodule Nous.HTTP.StreamBackend do
@moduledoc """
Behaviour for SSE / chunked streaming HTTP backends.
Implemented by `Nous.HTTP.StreamBackend.Req` (default) and
`Nous.HTTP.StreamBackend.Hackney`. Selection mirrors the non-streaming
`Nous.HTTP.Backend` resolution chain (per-call → env var → app config →
default). See `Nous.Providers.HTTP.stream/4` for the resolution order.
Pick a backend three ways, highest precedence first:
# 1. Per-call opt
Nous.Providers.HTTP.stream(url, body, headers,
stream_backend: Nous.HTTP.StreamBackend.Hackney)
# 2. Environment variable (req | hackney | "MyApp.MyStreamBackend")
export NOUS_HTTP_STREAM_BACKEND=hackney
# 3. Application config
config :nous, :http_stream_backend, Nous.HTTP.StreamBackend.Hackney
Default: `Nous.HTTP.StreamBackend.Req`.
## When to pick which
- `Nous.HTTP.StreamBackend.Req` — one HTTP stack across streaming and
non-streaming, simpler dependency story. Right default for most apps.
Backpressure is bounded by parsing speed, not by `stream_next/1`
pacing — a fast LLM + slow consumer can grow the consumer's mailbox.
Acceptable for typical LLM workloads where token rate is the
bottleneck.
- `Nous.HTTP.StreamBackend.Hackney` — strict pull-based backpressure
via `:hackney`'s `{:async, :once}` mode. The consumer paces the
producer chunk-by-chunk. Pick this when downstream consumers can
block per chunk (LiveView assigns + diff + push under load,
persistence-on-every-chunk, slow IO).
Both backends emit the same normalized event stream (parsed JSON maps,
`{:stream_done, reason}`, `{:stream_error, reason}`). Switching between
them does not require changes elsewhere.
## Custom backends
Implement `c:stream/4` and return `{:ok, Enumerable.t()}` where the
enumerable emits parsed JSON maps, `{:stream_done, reason}` tuples, or
`{:stream_error, reason}` tuples. The stream MUST halt after the first
`{:stream_error, _}` and after `{:stream_done, _}`.
"""
@doc """
Issue a streaming POST request and return a lazy `Enumerable.t()` of
parsed events.
## Options
* `:timeout` — receive timeout in milliseconds (default: `180_000`)
* `:connect_timeout` — TCP connect timeout in milliseconds (default: `30_000`)
* `:stream_parser` — module implementing `parse_buffer/1` for non-SSE
formats (e.g. JSON-array streams). Defaults to SSE.
Backends MAY accept additional options; unknown options should be ignored.
"""
@callback stream(
url :: String.t(),
body :: map(),
headers :: [{String.t(), String.t()}],
opts :: keyword()
) :: {:ok, Enumerable.t()} | {:error, term()}
end