Packages
fnord
0.9.40
0.9.40
0.9.39
0.9.38
0.9.37
0.9.36
0.9.35
0.9.34
0.9.33
0.9.32
0.9.31
0.9.30
0.9.29
0.9.28
0.9.27
0.9.26
0.9.25
0.9.24
0.9.23
0.9.22
0.9.21
0.9.20
0.9.19
0.9.18
0.9.17
0.9.16
0.9.15
0.9.14
0.9.13
0.9.12
0.9.11
0.9.10
0.9.9
0.9.8
0.9.7
0.9.6
0.9.5
0.9.4
0.9.3
0.9.2
0.9.1
0.9.0
0.8.99
0.8.98
0.8.97
0.8.96
0.8.95
0.8.94
0.8.93
0.8.92
0.8.91
0.8.90
0.8.89
0.8.88
0.8.87
0.8.86
0.8.85
0.8.84
0.8.83
0.8.82
0.8.81
0.8.80
0.8.79
0.8.78
0.8.77
0.8.76
0.8.75
0.8.74
0.8.73
0.8.72
0.8.71
0.8.70
0.8.69
0.8.68
0.8.67
0.8.66
0.8.65
0.8.64
0.8.63
0.8.62
0.8.61
0.8.60
0.8.59
0.8.58
0.8.57
0.8.56
0.8.55
0.8.54
0.8.53
0.8.52
0.8.51
0.8.50
0.8.49
0.8.48
0.8.47
0.8.46
0.8.45
0.8.44
0.8.43
0.8.42
0.8.41
0.8.40
0.8.39
0.8.38
0.8.37
0.8.36
0.8.35
0.8.34
0.8.33
0.8.32
0.8.31
0.8.30
0.8.29
0.8.27
0.8.26
0.8.25
0.8.24
0.8.23
0.8.22
0.8.21
0.8.20
0.8.19
0.8.18
0.8.17
0.8.16
0.8.15
0.8.14
0.8.13
0.8.12
0.8.11
0.8.1
0.8.0
0.7.24
0.7.23
0.7.22
0.7.21
0.7.20
0.7.19
0.7.18
0.7.17
0.7.16
0.7.15
0.7.14
0.7.13
0.7.12
0.7.11
0.7.10
0.7.9
0.7.8
0.7.7
0.7.6
0.7.5
0.7.3
0.7.2
0.7.1
0.7.0
0.6.9
0.6.8
0.6.7
0.6.6
0.6.5
0.6.4
0.6.3
0.6.1
0.6.0
0.5.9
0.5.8
0.5.7
0.5.6
0.5.5
0.5.4
0.5.3
0.5.2
0.5.1
0.5.0
0.4.44
0.4.43
0.4.42
0.4.41
0.4.40
0.4.39
0.4.38
0.4.37
0.4.36
0.4.35
0.4.34
0.4.33
0.4.32
0.4.30
0.4.29
0.4.28
0.4.27
0.4.26
0.4.25
0.4.24
0.4.23
0.4.22
0.4.21
0.4.20
0.4.19
0.4.18
0.4.17
0.4.16
0.4.15
0.4.14
0.4.13
0.4.12
0.4.11
0.4.10
0.4.9
0.4.8
0.4.7
0.4.6
0.4.5
0.4.4
0.4.3
0.4.2
0.4.1
0.4.0
0.3.0
0.2.0
0.1.0
AI code archaeology
Current section
Files
Jump to
Current section
Files
lib/ai/agent/review/reviewer.ex
defmodule AI.Agent.Review.Reviewer do
@moduledoc """
Master review agent. Coordinates a comprehensive post-implementation review
by researching the change, dispatching five specialist reviewers in parallel,
confirming their findings against the actual code, and producing a unified
severity-grouped report.
## Pipeline
1. **Formulate** - research the change and produce tailored prompts for each
specialist reviewer.
2. **Specialists** - five specialists (pedantic, acceptance, state flow,
no-slop, breadcrumbs) run in parallel, each producing structured findings
in their domain.
3. **Incorporate** - confirm each finding by reading the cited code, classify
as CONFIRMED/REJECTED/UNVERIFIABLE, and produce the final report.
"""
@behaviour AI.Agent
@behaviour AI.Agent.Composite
require Logger
@model AI.Model.smart()
# ---------------------------------------------------------------------------
# Prompts
# ---------------------------------------------------------------------------
@system_prompt """
You are the lead reviewer coordinating a comprehensive code review.
You are a STATIC ANALYSIS agent. You review code by reading it.
Do NOT run tests, linters, compilers, or any build commands.
Do NOT execute the code under review.
"""
@formulation_prompt """
Your first job is to understand the change and formulate targeted review prompts
for five specialist agents.
## Step 1: Understand the change
Your scope description may already include a git range and diff stat. If so,
use them directly - do NOT re-run `git diff --stat` or `git log`. Only run
git commands for data not already provided in your scope.
If the scope does NOT include a diff stat:
1. Run `git diff --stat` on the specified range to identify changed files.
2. Read the diffs for each changed file to understand what was modified.
In either case:
3. If design context was provided, use it. Otherwise, infer the design intent
from commit messages (`git log --oneline <range>`) and the code itself.
4. Note any aspects of the change that seem unusual, unrelated to the stated
purpose, or potentially risky.
5. Identify the application's **runtime model**: Is it a CLI tool (process
exits after each invocation)? A long-lived server (state persists across
requests)? A library (caller controls lifecycle)? A batch job? This
determines which classes of bugs are realistic - for example, "state not
cleaned up" is irrelevant in a short-lived process but critical in a
server. Include the runtime model in every specialist prompt.
## Step 2: Formulate specialist prompts
You have five specialists, each with a different focus:
1. **Pedantic** - mechanical correctness: spelling, naming, doc accuracy, specs,
guidelines, stale artifacts. Needs to know which files to read and what
conventions to check against.
2. **Acceptance** - behavioral/product review: UX coherency, behavioral delta,
integration effects, user assumptions. Needs to know the before-state and
the intended behavior so it can evaluate whether the change delivers.
3. **State Flow** - data flow and contracts: module boundaries, implicit FSMs,
error propagation, separation of concerns. Needs to know which module
boundaries to trace and what the expected data shapes are.
4. **NoSlop** - AI writing anti-pattern detection: change narration comments,
fourth wall breaks, em dashes, hedging language, filler phrases, stale
instruction artifacts. Needs to know which files to scan.
5. **BreadCrumbs** - comment narrative evaluation: do comments form a coherent
outline of the code's behavior? Do new modules/functions explain how they
fit into the larger system? Could a developer reconstruct the purpose from
comments alone? Needs the full file context, not just diffs.
For each specialist, write a prompt that:
- States the review scope (branch range, changed files)
- Provides relevant design context
- Highlights specific areas of concern you identified in Step 1
- Gives the specialist enough context to do focused, high-quality work
- Requires concrete reachability proof for any behavioral or state/data claim
- Requires an authoritative source of truth for the claim (caller contract,
producer, guideline, docs, behavior, or other owning layer)
- Requires a producer -> transform -> consumer chain for findings about state,
data shape, or cross-module behavior
- Calls out anything unusual (e.g., "3 of these changes appear to be bugfixes
not mentioned in the design - review these separately")
## Constraints
Your scope may begin with a "## Constraints" section listing contract/invariant constraints extracted from the change. Use them to filter out false positives and explicitly include which constraints a finding would violate.
Produce a JSON object with the specialist prompts and your scope summary.
"""
@aggregation_prompt """
You have received reports from five specialist reviewers. Your job now is to
verify their citations and produce a single, coherent report.
## Constraints usage
Group the final report output by constraints. Wherever applicable, reference the violated constraint ids in each confirmed finding. Use constraints to eliminate false positives that are disallowed by the stated contracts. In other words: group final report output by constraints.
## Verification process
Each specialist finding includes a `location` (file:line) and `evidence`
(quoted code). For each finding:
1. Read the cited file at the cited line to verify the evidence matches.
Only read the specific location - do NOT re-run `git diff --stat`,
`git log`, or other broad research commands. That work is already done.
2. Check whether the quoted code actually exists at that location and
whether the specialist's claim about its behavior is accurate.
3. Determine whether the finding is a new issue introduced by this change or a
pre-existing problem. Note pre-existing bugs identified separately.
4. Determine whether the finding can be realistically reproduced in normal
usage, if it requires unusual conditions, or if it's a technical or
theoretical flaw that is unlikely to manifest in practice.
5. Verify that the described bug can actually manifest given the
application's runtime model. A finding that requires conditions
impossible in the actual runtime context is not a real bug (e.g.,
state accumulation in a short-lived process, concurrency in
single-threaded code).
6. For any finding about state, data shape, cross-module contracts, or
behavior, prove the causal chain:
- Identify the authoritative producer or source of the state/data.
- Identify the transforms between producer and consumer.
- Identify the consumer or branch where failure occurs.
- Identify the real entrypoint/workflow that exercises this chain.
If the only way to trigger the issue is by manually fabricating invalid
data/state or bypassing the real producers/guards, reject the finding.
If the citation is real but you cannot prove the producer chain, classify
the finding as UNVERIFIABLE rather than CONFIRMED.
7. Verify intent. If the specialist flagged behavior as a bug but the code
appears to work as designed, check whether the behavior is intentional:
- Read the callers of the cited code to see if the pattern makes sense
in context.
- Check `git log` or `git blame` on the cited file for commit messages
explaining the decision.
- Use `memory_tool` (action=recall) and `prior_research` to search for
documented rationale.
If the behavior is intentional or an accepted limitation, reject the
finding.
8. Classify:
- **CONFIRMED**: The cited code matches, the claim is accurate, the bug
is reachable through realistic usage, you proved the workflow/producer
chain where applicable, and the behavior is not intentional.
- **REJECTED**: The citation is wrong, the claim is inaccurate, the bug
cannot manifest in the actual runtime context, or the behavior is an
intentional design decision (explain why briefly). Reject findings that
rely on manually seeded invalid state/data with no real producer path.
- **UNVERIFIABLE**: The citation is correct but you cannot confirm the
behavioral claim without deeper tracing (state what's missing). This is
the default for plausible claims that lack a proven trigger path,
producer chain, or authoritative source of truth.
If a citation clearly does not match the file contents (e.g., wrong line
numbers, code that doesn't exist, or content from a different branch), you
may do targeted investigation to determine what actually happened. This is
the ONLY situation where broader git commands are justified during
verification.
## Severity calibration
Assign final severity based on YOUR verification:
- **BLOCKING**: Incorrect behavior that will manifest in normal usage. You
confirmed the cited code behaves as the specialist described.
- **HIGH**: A real bug that requires specific but realistic conditions. You
verified the conditions are reachable from the cited location.
- **MEDIUM**: Edge cases, UX friction, or issues where the citation is correct
but the impact is limited or requires unusual conditions.
- **LOW**: Mechanical issues (stale docs, guideline violations, naming) that
don't affect correctness.
## Report format
### Scope
- Branch/range reviewed
- Design context (if provided)
### Confirmed findings
For each finding, grouped by severity (BLOCKING > HIGH > MEDIUM > LOW):
1. **Severity** and **category** (from the specialist's taxonomy)
2. **Source**: which specialist found it
3. **Location**: file:line
4. **Finding**: what the problem is
5. **Evidence**: the code you read to confirm it
6. **Trigger/Proof**: the workflow trigger, and for state/data/behavior
issues the producer -> transform -> consumer chain you verified
7. **Provenance**: branch-introduced or pre-existing
### Rejected findings (appendix, brief)
Findings you rejected and a one-line reason why.
### Pre-existing bugs (appendix, brief)
Findings you verified as real but pre-existing, with a one-line note on the
issue and its potential impact.
### Coverage gaps
Note which files or areas were NOT covered by any specialist.
"""
# ---------------------------------------------------------------------------
# Response formats
# ---------------------------------------------------------------------------
# OpenAI Responses-API structured-output strict mode requires the `required`
# array to list every key in `properties`. To express genuine optionality,
# use a nullable union type (e.g. `type: ["string", "null"]`) AND keep the
# key in required. design_context is documented as "Empty string if none"
# so it's always present in shape - no null needed.
@formulation_response_format %{
type: "json_schema",
json_schema: %{
name: "review_formulation",
schema: %{
type: "object",
required: [
"scope_summary",
"design_context",
"pedantic_prompt",
"acceptance_prompt",
"state_flow_prompt",
"no_slop_prompt",
"breadcrumbs_prompt"
],
additionalProperties: false,
properties: %{
scope_summary: %{
type: "string",
description:
"Brief summary of what the change does, which files are affected, and any design context."
},
design_context: %{
type: "string",
description:
"Design intent inferred from commits, docs, or caller-provided context. Empty string if none."
},
pedantic_prompt: %{
type: "string",
description: "Tailored review prompt for the pedantic specialist."
},
acceptance_prompt: %{
type: "string",
description: "Tailored review prompt for the acceptance specialist."
},
state_flow_prompt: %{
type: "string",
description: "Tailored review prompt for the state flow specialist."
},
no_slop_prompt: %{
type: "string",
description: "Tailored review prompt for the slop detection specialist."
},
breadcrumbs_prompt: %{
type: "string",
description: "Tailored review prompt for the comment narrative specialist."
}
}
}
}
}
@specialist_response_format %{
type: "json_schema",
json_schema: %{
name: "review_findings",
schema: %{
type: "object",
required: ["findings", "files_reviewed", "coverage_gaps"],
additionalProperties: false,
properties: %{
findings: %{
type: "array",
items: %{
type: "object",
required: [
"category",
"severity",
"location",
"description",
"evidence",
"trigger_scenario",
"reachability_analysis",
"source_of_truth",
"producer_chain",
"provenance"
],
additionalProperties: false,
properties: %{
category: %{
type: "string",
description:
"Finding category from the specialist's taxonomy (e.g. STALE, FRICTION, CONTRACT_MISMATCH)"
},
severity: %{
type: "string",
enum: ["BLOCKING", "HIGH", "MEDIUM", "LOW"]
},
location: %{
type: "string",
description:
"Exact file:line reference (e.g., 'lib/foo.ex:42'). Must match where you read the code."
},
description: %{type: "string", description: "What the problem is"},
evidence: %{
type: "string",
description:
"Exact code quoted from the cited location. Copy-paste from what you read - do not paraphrase."
},
trigger_scenario: %{
type: "string",
description:
"Concrete normal-usage trigger scenario. For purely mechanical findings, say why this is N/A."
},
reachability_analysis: %{
type: "string",
description:
"Why the issue is reachable in the real workflow, or why reachability is not relevant for this finding class."
},
source_of_truth: %{
type: "string",
description:
"Authoritative source for the claim: producer, caller contract, behavior, guideline, docs, or other owning layer."
},
producer_chain: %{
type: "string",
description:
"Producer -> transform -> consumer chain for state/data/behavior findings. Use 'N/A - mechanical finding' when not applicable."
},
provenance: %{
type: "string",
enum: ["branch-introduced", "pre-existing"]
}
}
}
},
files_reviewed: %{
type: "array",
items: %{type: "string"},
description: "List of files this specialist actually read"
},
coverage_gaps: %{
type: "array",
items: %{type: "string"},
description: "Files or areas in scope that were not reviewed"
}
}
}
}
}
@doc "The JSON schema response format used by specialist agents."
@spec specialist_response_format() :: map
def specialist_response_format, do: @specialist_response_format
@doc false
def __response_formats__,
do: [@formulation_response_format, @specialist_response_format]
# ---------------------------------------------------------------------------
# AI.Agent behaviour - entry point from the tool
# ---------------------------------------------------------------------------
@impl AI.Agent
def get_response(args) do
AI.Agent.Composite.run(__MODULE__, args)
end
# ---------------------------------------------------------------------------
# AI.Agent.Composite behaviour - step-based execution
# ---------------------------------------------------------------------------
@impl AI.Agent.Composite
def init(%{agent: agent, scope: scope}) do
tools = AI.Tools.basic_tools()
state = %AI.Agent.Composite{
agent: agent,
model: @model,
toolbox: tools,
request: scope,
response: nil,
error: nil,
messages: [
AI.Util.system_msg(AI.Util.project_context()),
AI.Util.system_msg(@system_prompt),
AI.Util.user_msg(scope)
],
internal: %{},
steps: [
AI.Agent.Composite.completion(:formulate, @formulation_prompt,
response_format: @formulation_response_format
),
[
AI.Agent.Composite.delegate(:pedantic, AI.Agent.Review.Pedantic, fn state ->
{:ok, prompts} = AI.Agent.Composite.get_state(state, :specialist_prompts)
%{prompt: prompts.pedantic_prompt, scope: state.request}
end),
AI.Agent.Composite.delegate(:acceptance, AI.Agent.Review.Acceptance, fn state ->
{:ok, prompts} = AI.Agent.Composite.get_state(state, :specialist_prompts)
%{prompt: prompts.acceptance_prompt, scope: state.request}
end),
AI.Agent.Composite.delegate(:state_flow, AI.Agent.Review.StateFlow, fn state ->
{:ok, prompts} = AI.Agent.Composite.get_state(state, :specialist_prompts)
%{prompt: prompts.state_flow_prompt, scope: state.request}
end),
AI.Agent.Composite.delegate(:no_slop, AI.Agent.Review.NoSlop, fn state ->
{:ok, prompts} = AI.Agent.Composite.get_state(state, :specialist_prompts)
%{prompt: prompts.no_slop_prompt, scope: state.request}
end),
AI.Agent.Composite.delegate(:breadcrumbs, AI.Agent.Review.BreadCrumbs, fn state ->
{:ok, prompts} = AI.Agent.Composite.get_state(state, :specialist_prompts)
%{prompt: prompts.breadcrumbs_prompt, scope: state.request}
end)
],
AI.Agent.Composite.completion(:incorporate, @aggregation_prompt)
]
}
{:ok, state}
end
@impl AI.Agent.Composite
def on_step_start(step, state) do
label =
case step.name do
:formulate -> "Researching change and formulating review prompts"
:pedantic -> "Dispatching pedantic reviewer"
:acceptance -> "Dispatching acceptance reviewer"
:state_flow -> "Dispatching state flow reviewer"
:no_slop -> "Dispatching slop detector"
:breadcrumbs -> "Dispatching comment narrative reviewer"
:incorporate -> "Confirming findings and producing final report"
end
UI.report_from(state.agent.name, label)
state
end
# Formulation uses a json_schema response_format, so the API guarantees
# schema conformance. A parse failure here indicates an API-level problem
# (refusal, network fault) rather than a malformed response. Halting is
# intentional: all specialist delegates pattern-match on :specialist_prompts,
# so continuing without valid prompts would crash downstream.
@impl AI.Agent.Composite
def on_step_complete(%{name: :formulate}, state) do
case SafeJson.decode_lenient(state.response, keys: :atoms!) do
{:ok,
%{
pedantic_prompt: _,
acceptance_prompt: _,
state_flow_prompt: _,
no_slop_prompt: _,
breadcrumbs_prompt: _
} = prompts} ->
AI.Agent.Composite.put_state(state, :specialist_prompts, prompts)
other ->
Logger.warning("Reviewer: formulation parse failed: #{inspect(other)}")
%{state | error: "Failed to parse formulation response"}
end
end
def on_step_complete(_step, state), do: state
@impl AI.Agent.Composite
def get_next_steps(_step, _state), do: []
# Halt on any step failure. Retry is not useful here: formulation failures
# indicate API-level problems, and specialist/incorporation failures mean
# the conversation state is already inconsistent.
@impl AI.Agent.Composite
def on_error(_step, _error, state) do
{:halt, state}
end
@doc "Return internal prompts (test helper)."
@spec __send__(:formulation_prompt | :aggregation_prompt) :: String.t()
def __send__(:formulation_prompt), do: @formulation_prompt
def __send__(:aggregation_prompt), do: @aggregation_prompt
end