Current section

Files

Jump to
cmdc lib cmdc telemetry.ex
Raw

lib/cmdc/telemetry.ex

defmodule CMDC.Telemetry do
@moduledoc """
CMDC 标准 `:telemetry` 事件契约。
CMDC 核心库只**广播事件**,不内嵌任何 sink;用户可挂任意 handler
到 Langfuse / LangSmith / Grafana Tempo / Datadog / Splunk 等。
## 设计原则
1. **零强制依赖**`:telemetry` 已是 BEAM 生态事实标准,CMDC 仅广播
2. **事件命名稳定**:锁定 schema,向后兼容
3. **metadata 白名单**:明确列出 metadata 字段,避免泄露敏感数据
4. **与现有 EventBus 互补**:EventBus 走 PubSub 给业务订阅者,
Telemetry 走 handler 给可观测性栈
## 事件清单(v0.5.1 共 18 事件 = 6 旧 + 10 v0.5 + 2 v0.5.1)
### v0.4 已有 6 事件(走 `EventBus → bridge_event/2` 桥接)
| 事件名 | 触发时机 | metadata |
|---|---|---|
| `[:cmdc, :agent, :turn, :start]` | Agent 单 turn 开始 | session_id, turn |
| `[:cmdc, :agent, :turn, :stop]` | Agent 单 turn 结束 | session_id, turn, duration_ms, outcome |
| `[:cmdc, :llm, :request, :start]` | LLM 调用前 | session_id, turn, model |
| `[:cmdc, :llm, :request, :stop]` | LLM 调用后 | session_id, turn, model, tokens_in, tokens_out, duration_ms |
| `[:cmdc, :tool, :exec, :start]` | 工具开始执行 | session_id, tool, call_id |
| `[:cmdc, :tool, :exec, :stop]` | 工具结束 | session_id, tool, call_id, duration_ms, error? |
### v0.5 新增 10 事件(直接 `:telemetry.execute` 埋点,不经 EventBus)
这一类事件**不走 EventBus**——它们是 system-wide 可观测性事件,不是业务订阅事件,
避免 per-session 订阅者产生噪音。Studio / Hive 等集成方 attach 这些事件接入 Langfuse /
Datadog / Prometheus 即可。
| 事件名 | 触发时机 | measurements / metadata |
|---|---|---|
| `[:cmdc, :plugin, :pipeline, :start]` | Plugin Pipeline 整个 hook 开始 | `%{system_time}` / `%{session_id, hook, plugin_count}` |
| `[:cmdc, :plugin, :pipeline, :stop]` | Plugin Pipeline 整个 hook 完成(**hook 粒度,非 per-plugin**| `%{duration_ms}` / `%{session_id, hook, halted_by, action}` |
| `[:cmdc, :plugin, :crash]` | 单个 plugin handle_event 抛出异常(已被 Pipeline rescue) | `%{count: 1}` / `%{session_id, plugin, hook, kind, reason}` |
| `[:cmdc, :compactor, :run, :start]` | Compactor 开始压缩 | `%{system_time, tokens_before}` / `%{session_id}` |
| `[:cmdc, :compactor, :run, :stop]` | Compactor 完成 | `%{duration_ms, tokens_after, ratio}` / `%{session_id, strategy, outcome}` |
| `[:cmdc, :checkpoint, :save, :stop]` | Checkpoint.save/3 完成 | `%{duration_ms, snapshot_bytes}` / `%{session_id, backend, label, outcome}` |
| `[:cmdc, :checkpoint, :load, :stop]` | Checkpoint.load/2 完成 | `%{duration_ms, snapshot_bytes, hit?}` / `%{session_id, backend, outcome}` |
| `[:cmdc, :subagent, :spawn, :start]` | SubAgent.Supervisor 启动子 Agent | `%{system_time}` / `%{parent_session_id, sub_session_id, name}` |
| `[:cmdc, :subagent, :spawn, :stop]` | 子 Agent 结束 | `%{duration_ms}` / `%{parent_session_id, sub_session_id, outcome}` |
| `[:cmdc, :agent, :hibernate, :configured]` | Agent init 时配置了 `hibernate_after_ms` | `%{ms}` / `%{session_id, enabled?}` |
> **注意**:OTP `:gen_statem``:hibernate_after` 是被动行为,无原生 enter/wake 事件挂钩,
> 因此 v0.5 只暴露 `:configured` 一次性事件(启动时 emit),用于观测启用率。
> 真实 hibernate 命中率请通过 `Process.info(pid, :memory)` 周期采样推算。
### v0.5.1 新增 2 事件(Provider Registry 命中率 / 注册量监控)
| 事件名 | 触发时机 | measurements / metadata |
|---|---|---|
| `[:cmdc, :provider, :registry, :lookup]` | `CMDC.Provider.Registry.lookup/1` 完成 | `%{duration_us}` / `%{name, hit?}` |
| `[:cmdc, :provider, :registry, :register]` | `CMDC.Provider.Registry.register/2` 完成 | `%{}` / `%{name, broadcaster_called?}` |
> **`hit?` 字段**:lookup 命中返 `true`、未命中返 `false`。多租户场景可
> 接入 Prometheus / Datadog 监控"profile 未注册" 漂移;典型告警:
> 5 分钟内 `hit? == false` 占比 > 1% 表示集成方持久层与 Registry 失同步。
## 与 :tool_execution_metrics 业务事件的关系
`:tool_execution_metrics` 通过 EventBus 广播业务事件(per-session 订阅)。
Telemetry 通过 `:telemetry.execute/3` 广播 `[:cmdc, :tool, :exec, :*]` 全局
事件(system-wide 监控)。两者数据字段一致,触发点合并在 Agent 工具执行器内。
## 实现说明
- **emitter 桥接模式**:Agent 内部广播器在广播 EventBus 事件的同时,
把 6 个核心事件映射到 telemetry(无需改 ToolRunner / Provider)
- **schema 版本号**:所有事件 metadata 含 `schema_version: 1`
- 用户用 `:telemetry.attach/4``CMDC.Telemetry.attach_logger/0` 接入
## Quick Start
# 1. attach 默认 logger(开发期)
CMDC.Telemetry.attach_logger()
# 2. attach 自定义 handler(接入 Langfuse)
:telemetry.attach_many(
"my-langfuse-handler",
CMDC.Telemetry.all_events(),
&MyApp.LangfuseHandler.handle/4,
nil
)
# 3. detach
:telemetry.detach("my-langfuse-handler")
完整可观测性配方见 `docs/recipes/observability/`(v0.4.0 提供 Langfuse 示例)。
"""
require Logger
@schema_version 1
@v04_events [
[:cmdc, :agent, :turn, :start],
[:cmdc, :agent, :turn, :stop],
[:cmdc, :llm, :request, :start],
[:cmdc, :llm, :request, :stop],
[:cmdc, :tool, :exec, :start],
[:cmdc, :tool, :exec, :stop]
]
@v05_events [
[:cmdc, :plugin, :pipeline, :start],
[:cmdc, :plugin, :pipeline, :stop],
[:cmdc, :plugin, :crash],
[:cmdc, :compactor, :run, :start],
[:cmdc, :compactor, :run, :stop],
[:cmdc, :checkpoint, :save, :stop],
[:cmdc, :checkpoint, :load, :stop],
[:cmdc, :subagent, :spawn, :start],
[:cmdc, :subagent, :spawn, :stop],
[:cmdc, :agent, :hibernate, :configured]
]
@v05_1_events [
[:cmdc, :provider, :registry, :lookup],
[:cmdc, :provider, :registry, :register]
]
@events @v04_events ++ @v05_events ++ @v05_1_events
# ==========================================================================
# Public API
# ==========================================================================
@doc "返回 CMDC.Telemetry 所有事件列表。"
@spec all_events() :: [[atom()]]
def all_events, do: @events
@doc "返回 telemetry schema 版本号。"
@spec schema_version() :: pos_integer()
def schema_version, do: @schema_version
@doc """
执行 telemetry 事件 — 内部桥接使用。
自动加 `schema_version: 1` 到 metadata。
"""
@spec execute([atom()], map(), map()) :: :ok
def execute(event_name, measurements, metadata) when is_list(event_name) do
enriched = Map.put(metadata, :schema_version, @schema_version)
:telemetry.execute(event_name, measurements, enriched)
end
@doc """
附加一个开发期 Logger handler,把所有 CMDC telemetry 事件打到 console。
生产环境请用 `:telemetry.attach_many/4` 接入实际可观测性栈。
"""
@spec attach_logger(keyword()) :: :ok | {:error, term()}
def attach_logger(opts \\ []) do
handler_id = Keyword.get(opts, :handler_id, "cmdc-telemetry-logger")
:telemetry.attach_many(handler_id, all_events(), &__MODULE__.log_event/4, nil)
end
@doc "Detach Logger handler。"
@spec detach_logger(keyword()) :: :ok | {:error, :not_found}
def detach_logger(opts \\ []) do
handler_id = Keyword.get(opts, :handler_id, "cmdc-telemetry-logger")
:telemetry.detach(handler_id)
end
@doc false
def log_event(event_name, measurements, metadata, _config) do
Logger.debug(fn ->
"[CMDC.Telemetry] #{inspect(event_name)} measurements=#{inspect(measurements)} metadata=#{inspect(Map.delete(metadata, :schema_version))}"
end)
end
# ==========================================================================
# 内部事件桥接 (来自 Agent.Emitter)
# ==========================================================================
@doc """
从 EventBus 事件桥接到 telemetry。
`Agent.Emitter.broadcast/2` 调用。映射规则:
| EventBus 事件 | Telemetry 事件 |
|---|---|
| `{:agent_start, session_id, turn}` | `[:cmdc, :agent, :turn, :start]` |
| `{:agent_end, payload}` (success) | `[:cmdc, :agent, :turn, :stop]` |
| `{:llm_request_start, ...}` | `[:cmdc, :llm, :request, :start]` |
| `{:llm_response, payload}` | `[:cmdc, :llm, :request, :stop]` |
| `{:tool_started, name, call_id, ...}` | `[:cmdc, :tool, :exec, :start]` |
| `{:tool_ended, name, call_id, ...}` | `[:cmdc, :tool, :exec, :stop]` |
| `{:tool_execution_metrics, ...}` | `[:cmdc, :tool, :exec, :stop]`(兼容路径)|
其他事件不映射(继续走 EventBus)。
"""
@spec bridge_event(String.t(), term()) :: :ok
def bridge_event(session_id, event) do
case map_event(session_id, event) do
nil ->
:ok
{name, measurements, metadata} ->
execute(name, measurements, metadata)
end
end
# ==========================================================================
# 私有:事件映射
# ==========================================================================
defp map_event(session_id, event) do
case event do
{:agent_start, sid, turn} ->
{[:cmdc, :agent, :turn, :start], %{system_time: System.system_time(:millisecond)},
%{session_id: sid, turn: turn}}
{:agent_end, payload} when is_map(payload) ->
{[:cmdc, :agent, :turn, :stop], %{duration_ms: Map.get(payload, :duration_ms, 0)},
%{
session_id: session_id,
turn: Map.get(payload, :turn),
outcome: Map.get(payload, :outcome, :finished)
}}
{:llm_request_start, sid, turn, model} ->
{[:cmdc, :llm, :request, :start], %{system_time: System.system_time(:millisecond)},
%{session_id: sid, turn: turn, model: model}}
{:llm_response, payload} when is_map(payload) ->
{[:cmdc, :llm, :request, :stop],
%{
duration_ms: Map.get(payload, :duration_ms, 0),
tokens_in: Map.get(payload, :prompt_tokens, 0),
tokens_out: Map.get(payload, :completion_tokens, 0)
}, %{session_id: session_id, model: Map.get(payload, :model)}}
{:tool_started, name, call_id, _args} ->
{[:cmdc, :tool, :exec, :start], %{system_time: System.system_time(:millisecond)},
%{session_id: session_id, tool: name, call_id: call_id}}
{:tool_ended, name, call_id, payload} ->
{[:cmdc, :tool, :exec, :stop], %{duration_ms: Map.get(payload, :duration_ms, 0)},
%{
session_id: session_id,
tool: name,
call_id: call_id,
error?: Map.get(payload, :error?, false)
}}
{:tool_execution_metrics, payload} when is_map(payload) ->
# 兼容路径:把业务事件也桥到 telemetry
{[:cmdc, :tool, :exec, :stop], %{duration_ms: Map.get(payload, :duration_ms, 0)},
%{
session_id: session_id,
tool: Map.get(payload, :tool),
call_id: Map.get(payload, :call_id),
error?: Map.get(payload, :error?, false)
}}
_ ->
nil
end
end
end