Packages

The unified observability layer for the AI Control Plane, delivering full-fidelity tracing for AI agent reasoning, tool calls, and state transitions.

Current section

2 Versions

Jump to

Compare versions

17 files changed
+1553 additions
-81 deletions
  @@ -1,4 +0,0 @@
1 - # Used by "mix format"
2 - [
3 - inputs: ["{mix,.formatter}.exs", "{config,lib,test}/**/*.{ex,exs}"]
4 - ]
  @@ -0,0 +1,15 @@
1 + # Changelog
2 +
3 + All notable changes to this project will be documented in this file.
4 +
5 + The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
6 +
7 + ## [0.1.0] - 2025-10-29
8 +
9 + ### Added
10 + - First public release establishing AITrace as an Elixir-native observability layer for AI agent workflows.
11 + - Macro-based tracing DSL (`AITrace.trace/2`, `AITrace.span/2`) that captures nested spans and restores context across blocks.
12 + - Structured event and attribute recording with `AITrace.add_event/2` and `AITrace.with_attributes/1`.
13 + - Context propagation utilities and process dictionary helpers to correlate spans within complex control flow.
14 + - Pluggable exporter pipeline with Console and File exporters for immediate inspection or archival of traces.
15 + - Collector process and data model for Traces, Spans, and Events, designed to support future backends and real-time streaming.
  @@ -0,0 +1,21 @@
1 + MIT License
2 +
3 + Copyright (c) 2025 nshkrdotcom <ZeroTrust@NSHkr.com>
4 +
5 + Permission is hereby granted, free of charge, to any person obtaining a copy
6 + of this software and associated documentation files (the "Software"), to deal
7 + in the Software without restriction, including without limitation the rights
8 + to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9 + copies of the Software, and to permit persons to whom the Software is
10 + furnished to do so, subject to the following conditions:
11 +
12 + The above copyright notice and this permission notice shall be included in all
13 + copies or substantial portions of the Software.
14 +
15 + THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16 + IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17 + FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18 + AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19 + LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20 + OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21 + SOFTWARE.
\ No newline at end of file
  @@ -1,5 +1,9 @@
1 1 # AITrace
2 2
3 + <p align="center">
4 + <img src="assets/ai_trace.svg" alt="AITrace logo" width="220" />
5 + </p>
6 +
3 7 > The unified observability layer for the AI Control Plane.
4 8
5 9 `AITrace` provides the unified observability layer for the AI Control Plane, transforming opaque, non-deterministic AI processes into fully interpretable and debuggable execution traces.
  @@ -31,65 +35,212 @@ Debugging an AI agent is fundamentally different. It is like performing forensic
31 35
32 36 * **Context:** An immutable Elixir map (`%AITrace.Context{}`) that carries the `trace_id` and the current `span_id`. This context is explicitly passed through the entire call stack of a traced operation, ensuring all telemetry is correctly correlated.
33 37
34 - ## Architectural Design & API
38 + ## Installation
35 39
36 - `AITrace` is designed as a set of ergonomic Elixir macros and functions that make instrumentation feel natural.
37 -
38 - ### The Core API
40 + Add `aitrace` to your `mix.exs` dependencies:
39 41
40 42 ```elixir
41 - # lib/my_app/agent.ex
43 + def deps do
44 + [
45 + {:aitrace, "~> 0.1.0"}
46 + ]
47 + end
48 + ```
42 49
43 - def handle_user_message(message, state) do
44 - # 1. Start a new trace for the entire transaction
45 - AITrace.trace "agent.handle_message" do
46 - # The `trace` macro injects a `ctx` variable into the scope.
47 -
48 - # 2. Add point-in-time events with rich metadata.
49 - AITrace.add_event(ctx, "Initial state loaded", %{agent_id: state.id})
50 + ## Quick Start
50 51
51 - # 3. Wrap discrete, timed operations in spans.
52 - {:ok, response, new_ctx} = AITrace.span ctx, "reasoning_loop" do
53 - # The `span` macro also yields a new, updated context.
54 - DSPex.execute(reasoning_logic, %{message: message}, context: new_ctx)
52 + ```elixir
53 + defmodule MyApp.Agent do
54 + require AITrace # Required to use the macros
55 +
56 + def handle_user_message(message, state) do
57 + # 1. Start a new trace for the entire transaction
58 + AITrace.trace "agent.handle_message" do
59 + # 2. Add point-in-time events with rich metadata
60 + AITrace.add_event("request_received", %{message_length: String.length(message)})
61 +
62 + # 3. Wrap discrete, timed operations in spans
63 + response = AITrace.span "reasoning_loop" do
64 + # Add attributes to the current span
65 + AITrace.with_attributes(%{model: "gpt-4", temperature: 0.7})
66 +
67 + # Perform reasoning
68 + think_about(message)
69 + end
70 +
71 + AITrace.add_event("reasoning_complete", %{token_usage: response.tokens})
72 +
73 + {:reply, response.answer, update_state(state)}
55 74 end
56 -
57 - # new_ctx now contains the updated span information.
58 - AITrace.add_event(new_ctx, "Reasoning complete", %{token_usage: response.token_usage})
59 -
60 - {:reply, response.answer, update_state(state)}
61 75 end
62 76 end
63 77 ```
64 78
65 - ### Key Design Points
79 + ## Core API
66 80
67 - * **Context Propagation:** The biggest challenge is passing the `ctx`. The API will provide helpers and patterns (like using a `with` block) to make this propagation clear and explicit, avoiding "magic" context passing.
68 - * **OTP Integration:** The library will include helpers for stashing and retrieving the `AITrace` context in `GenServer` calls and `Oban` jobs, making it easy to continue a trace across process boundaries.
69 - * **Pluggable Backends (Exporters):** `AITrace` itself is only responsible for generating telemetry data. It will use a configurable "Exporter" to send this data somewhere. This makes the system incredibly flexible.
70 - * `AITrace.Exporter.Console`: Prints human-readable traces to the terminal for local development.
71 - * `AITrace.Exporter.File`: Dumps structured JSON traces to a file.
72 - * `AITrace.Exporter.Phoenix`: (Future) A backend that sends traces over Phoenix Channels to a live UI.
73 - * `AITrace.Exporter.OpenTelemetry`: (Future) A backend that converts `AITrace` data into OTel format for integration with existing systems like Jaeger or Honeycomb.
81 + ### Starting a Trace
74 82
75 - ## Integration with the Ecosystem
83 + ```elixir
84 + AITrace.trace "operation_name" do
85 + # Your code here - context is stored in process dictionary
86 + end
87 + ```
76 88
77 - `AITrace` is the glue that binds the entire portfolio into a single, observable system.
89 + ### Creating Spans
78 90
79 - * **`DSPex` Instrumentation:** `DSPex` will be modified to accept an optional `AITrace.Context`. When provided, it will automatically create spans for `llm_call`, `prompt_render`, and `output_parsing`. It will add events for which modules are being used (`Predict`, `ChainOfThought`) and log the full, raw prompt/completion data as span attributes.
91 + ```elixir
92 + AITrace.span "span_name" do
93 + # Timed operation - duration is automatically measured
94 + end
95 + ```
80 96
81 - * **`Altar` Instrumentation:** The host application's tool-execution wrapper (as designed in Architecture A) will use `AITrace.span` to wrap every call to `Altar.LATER.Executor`. The span's attributes will include the tool's name, arguments, and the validated result or error. This provides perfect observability into tool usage.
97 + ### Adding Events
82 98
83 - * **`Snakepit` Instrumentation:** `Snakepit` will be enhanced to accept an `AITrace.Context`. It will extract the `trace_id` and `span_id` and pass them as gRPC metadata to the Python worker. A corresponding Python library will then be able to reconstruct the trace context on the other side, enabling true cross-language distributed tracing. The duration of the gRPC call itself will be captured in a span.
99 + ```elixir
100 + AITrace.add_event("event_name", %{key: "value"})
101 + AITrace.add_event("simple_event") # No attributes
102 + ```
84 103
85 - ## The End Goal: The "Execution Cinema"
104 + ### Adding Attributes
86 105
87 - The data generated by `AITrace` is structured to power a rich, interactive debugging UI. This UI is the ultimate user-facing product of the library.
106 + ```elixir
107 + AITrace.with_attributes(%{user_id: 42, region: "us-west"})
108 + ```
88 109
89 - ### Features
90 - * **Waterfall View:** A visual timeline showing the nested spans of a trace, allowing a developer to immediately spot long-running operations.
91 - * **Context Explorer:** Clicking on any span reveals its full attributes—the exact prompt sent to an LLM, the JSON returned from a tool, the error message from a validation failure.
92 - * **State Diff:** For spans that include agent state changes, the UI can show a "diff" of the state before and after the operation.
93 - * **Causal Flow:** The UI will clearly visualize the flow of data and control, making it easy to follow the agent's "train of thought."
110 + ### Accessing Context
94 111
95 - This is not just a log viewer; it is a purpose-built, interactive debugger for AI reasoning.
\ No newline at end of file
112 + ```elixir
113 + ctx = AITrace.get_current_context()
114 + IO.inspect(ctx.trace_id)
115 + IO.inspect(ctx.span_id)
116 + ```
117 +
118 + ## Configuration
119 +
120 + Configure exporters in your application config:
121 +
122 + ```elixir
123 + # config/config.exs
124 + config :aitrace,
125 + exporters: [
126 + {AITrace.Exporter.Console, verbose: true, color: true},
127 + {AITrace.Exporter.File, directory: "./traces"}
128 + ]
129 + ```
130 +
131 + ### Available Exporters
132 +
133 + * **`AITrace.Exporter.Console`** - Prints human-readable traces to stdout
134 + - Options: `verbose` (show attributes/events), `color` (ANSI colors)
135 +
136 + * **`AITrace.Exporter.File`** - Writes JSON traces to files
137 + - Options: `directory` (output directory, default: "./traces")
138 +
139 + ### Creating Custom Exporters
140 +
141 + Implement the `AITrace.Exporter` behavior:
142 +
143 + ```elixir
144 + defmodule MyApp.CustomExporter do
145 + @behaviour AITrace.Exporter
146 +
147 + @impl true
148 + def init(opts), do: {:ok, opts}
149 +
150 + @impl true
151 + def export(trace, state) do
152 + # Send trace to your backend
153 + IO.inspect(trace)
154 + {:ok, state}
155 + end
156 +
157 + @impl true
158 + def shutdown(_state), do: :ok
159 + end
160 + ```
161 +
162 + ## Examples
163 +
164 + See `examples/basic_usage.exs` for a complete working example:
165 +
166 + ```bash
167 + mix run examples/basic_usage.exs
168 + ```
169 +
170 + Output:
171 + ```
172 + Trace: b37b73325dbd626481e0ff3e89de02c8
173 + â–¸ reasoning (10.84ms) âś“
174 + Attributes: %{model: "gpt-4", temperature: 0.7}
175 + • reasoning_complete
176 + %{thought_count: 3}
177 + â–¸ tool_execution (5.95ms) âś“
178 + Attributes: %{tool: "web_search"}
179 + â–¸ response_generation (8.98ms) âś“
180 + Attributes: %{tokens: 150}
181 + ```
182 +
183 + ## Architecture
184 +
185 + ### Data Model
186 +
187 + - **AITrace.Context** - Carries trace_id and span_id through the call stack
188 + - **AITrace.Span** - Timed operations with start/end times, attributes, and events
189 + - **AITrace.Event** - Point-in-time annotations within spans
190 + - **AITrace.Trace** - Complete trace containing all spans
191 +
192 + ### Runtime
193 +
194 + - **AITrace.Collector** - In-memory Agent storing active traces
195 + - **AITrace.Application** - Supervision tree managing the collector
196 + - Context stored in process dictionary for implicit propagation
197 +
198 + ### Future Integrations
199 +
200 + `AITrace` is designed to integrate with other AI infrastructure:
201 +
202 + * **DSPex** - Automatic instrumentation for LLM calls and prompt rendering
203 + * **Altar** - Tool execution tracing with arguments and results
204 + * **Snakepit** - Cross-language tracing via gRPC metadata
205 + * **Phoenix Channels** - Real-time trace streaming to web UIs
206 + * **OpenTelemetry** - Export to standard observability platforms
207 +
208 + ## Development Status
209 +
210 + **âś… Implemented (v0.1.0)**
211 + - Core data structures (Context, Span, Event, Trace)
212 + - Trace and span macros with automatic timing
213 + - Event and attribute APIs
214 + - Console exporter (human-readable output)
215 + - File exporter (JSON format)
216 + - Comprehensive test suite (80 tests)
217 + - Working examples
218 +
219 + **đźš§ Planned**
220 + - Phoenix Channel exporter for real-time streaming
221 + - OpenTelemetry exporter
222 + - OTP integration helpers (GenServer, Oban)
223 + - Cross-process context propagation
224 + - "Execution Cinema" web UI with waterfall views
225 + - DSPex, Altar, and Snakepit integrations
226 +
227 + ## Testing
228 +
229 + ```bash
230 + # Run all tests
231 + mix test
232 +
233 + # Run with coverage
234 + mix test --cover
235 +
236 + # Run example
237 + mix run examples/basic_usage.exs
238 + ```
239 +
240 + ## License
241 +
242 + MIT - See [LICENSE](LICENSE) for details.
243 +
244 + ## Contributing
245 +
246 + AITrace is part of the AI Control Plane ecosystem. Contributions welcome!
  @@ -0,0 +1,160 @@
1 + <svg width="384" height="432" viewBox="0 0 100 112" xmlns="http://www.w3.org/2000/svg" version="1.1">
2 + <defs>
3 + <linearGradient id="hex-fill" x1="5%" y1="0%" x2="95%" y2="100%">
4 + <stop offset="0%" stop-color="#0d0b18"/>
5 + <stop offset="42%" stop-color="#181f3b"/>
6 + <stop offset="100%" stop-color="#0a5365"/>
7 + </linearGradient>
8 +
9 + <linearGradient id="hex-stroke" x1="20%" y1="0%" x2="80%" y2="100%">
10 + <stop offset="0%" stop-color="#7dd9ff"/>
11 + <stop offset="40%" stop-color="#6a8cff"/>
12 + <stop offset="100%" stop-color="#a96bff"/>
13 + </linearGradient>
14 +
15 + <radialGradient id="hex-core" cx="50%" cy="42%" r="65%">
16 + <stop offset="0%" stop-color="#00ffe5" stop-opacity="0.28"/>
17 + <stop offset="45%" stop-color="#203d7a" stop-opacity="0.6"/>
18 + <stop offset="100%" stop-color="#070b1c" stop-opacity="0.2"/>
19 + </radialGradient>
20 +
21 + <linearGradient id="band-gradient" x1="0%" y1="0%" x2="100%" y2="0%">
22 + <stop offset="0%" stop-color="#21a0ff" stop-opacity="0"/>
23 + <stop offset="40%" stop-color="#21a0ff" stop-opacity="0.42"/>
24 + <stop offset="60%" stop-color="#d95cff" stop-opacity="0.48"/>
25 + <stop offset="100%" stop-color="#d95cff" stop-opacity="0"/>
26 + </linearGradient>
27 +
28 + <linearGradient id="trace-stroke" x1="0%" y1="0%" x2="100%" y2="100%">
29 + <stop offset="0%" stop-color="#5dffd2"/>
30 + <stop offset="45%" stop-color="#6ab0ff"/>
31 + <stop offset="100%" stop-color="#ab66ff"/>
32 + </linearGradient>
33 +
34 + <radialGradient id="node-glow" cx="50%" cy="50%" r="50%">
35 + <stop offset="0%" stop-color="#ffffff" stop-opacity="0.95"/>
36 + <stop offset="40%" stop-color="#7eefef" stop-opacity="0.8"/>
37 + <stop offset="100%" stop-color="#55c7ff" stop-opacity="0.05"/>
38 + </radialGradient>
39 +
40 + <filter id="outer-glow" x="-20%" y="-20%" width="140%" height="140%">
41 + <feGaussianBlur in="SourceAlpha" stdDeviation="1.8" result="blur"/>
42 + <feColorMatrix in="blur" type="matrix"
43 + values="0 0 0 0 0.22
44 + 0 0 0 0 0.54
45 + 0 0 0 0 0.94
46 + 0 0 0 0.35 0"
47 + />
48 + <feMerge>
49 + <feMergeNode/>
50 + <feMergeNode in="SourceGraphic"/>
51 + </feMerge>
52 + </filter>
53 +
54 + <filter id="trace-glow" x="-35%" y="-35%" width="170%" height="170%">
55 + <feGaussianBlur in="SourceGraphic" stdDeviation="0.8" result="soft"/>
56 + <feMerge>
57 + <feMergeNode in="soft"/>
58 + <feMergeNode in="SourceGraphic"/>
59 + </feMerge>
60 + </filter>
61 +
62 + <mask id="band-mask">
63 + <rect x="0" y="0" width="100" height="112" fill="#fff"/>
64 + <path d="M 50 0 L 93.3 25 L 93.3 75 L 50 100 L 6.7 75 L 6.7 25 Z"
65 + fill="black" stroke="black" stroke-width="12"
66 + transform="translate(0, 4)"/>
67 + </mask>
68 +
69 + <clipPath id="hex-clip">
70 + <path d="M 50 4 L 93.3 29 L 93.3 79 L 50 104 L 6.7 79 L 6.7 29 Z"/>
71 + </clipPath>
72 + </defs>
73 +
74 + <g filter="url(#outer-glow)">
75 + <path
76 + d="M 50 4 L 93.3 29 L 93.3 79 L 50 104 L 6.7 79 L 6.7 29 Z"
77 + fill="url(#hex-fill)"
78 + stroke="url(#hex-stroke)"
79 + stroke-width="1.8"
80 + stroke-linejoin="round"
81 + />
82 + </g>
83 +
84 + <g clip-path="url(#hex-clip)">
85 + <rect x="0" y="0" width="100" height="112" fill="url(#hex-core)"/>
86 +
87 + <rect x="-40" y="22" width="180" height="28" fill="url(#band-gradient)" opacity="0.7"
88 + transform="rotate(-7 50 50)" mask="url(#band-mask)"/>
89 +
90 + <path d="M 50 4 L 93.3 29 L 93.3 79 L 50 104 L 6.7 79 L 6.7 29 Z"
91 + fill="none"
92 + stroke="rgba(109, 225, 255, 0.18)"
93 + stroke-width="0.8"
94 + />
95 + </g>
96 +
97 + <g clip-path="url(#hex-clip)" filter="url(#trace-glow)">
98 + <path
99 + d="M 18 74 C 28 48, 38 36, 50 46 S 75 82, 88 60"
100 + fill="none"
101 + stroke="url(#trace-stroke)"
102 + stroke-width="2.4"
103 + stroke-linecap="round"
104 + stroke-linejoin="round"
105 + />
106 +
107 + <path
108 + d="M 28 86 C 42 70, 59 66, 69 55 C 78 45, 82 33, 89 32"
109 + fill="none"
110 + stroke="rgba(121, 216, 255, 0.6)"
111 + stroke-width="1.4"
112 + stroke-linecap="round"
113 + />
114 +
115 + <path
116 + d="M 20 48 Q 36 58, 48 50 T 70 30"
117 + fill="none"
118 + stroke="rgba(165, 106, 255, 0.55)"
119 + stroke-width="1"
120 + stroke-linecap="round"
121 + stroke-dasharray="2.6 2.6"
122 + />
123 + </g>
124 +
125 + <g clip-path="url(#hex-clip)">
126 + <path
127 + d="M 39 86 L 50 44 L 61.4 86"
128 + fill="none"
129 + stroke="rgba(123, 222, 255, 0.42)"
130 + stroke-width="3.2"
131 + stroke-linecap="round"
132 + stroke-linejoin="round"
133 + />
134 + <path
135 + d="M 44.5 68 H 55.5"
136 + fill="none"
137 + stroke="rgba(202, 146, 255, 0.45)"
138 + stroke-width="3.2"
139 + stroke-linecap="round"
140 + />
141 + <path
142 + d="M 57 34 V 84"
143 + fill="none"
144 + stroke="rgba(102, 206, 255, 0.38)"
145 + stroke-width="2"
146 + stroke-linecap="round"
147 + stroke-dasharray="3.4 3.4"
148 + />
149 + </g>
150 +
151 + <g clip-path="url(#hex-clip)">
152 + <circle cx="50" cy="46" r="3.2" fill="url(#node-glow)" opacity="0.92"/>
153 + <circle cx="75" cy="72" r="2.4" fill="url(#node-glow)" opacity="0.82"/>
154 + <circle cx="31" cy="76" r="2.8" fill="url(#node-glow)" opacity="0.88"/>
155 + <circle cx="68" cy="40" r="2.1" fill="url(#node-glow)" opacity="0.8"/>
156 + <circle cx="83" cy="46" r="2.6" fill="url(#node-glow)" opacity="0.86"/>
157 + <circle cx="43" cy="88" r="1.9" fill="url(#node-glow)" opacity="0.75"/>
158 + <circle cx="57" cy="86" r="2.2" fill="url(#node-glow)" opacity="0.78"/>
159 + </g>
160 + </svg>
Loading more files…