Packages

development-driven-specs — the dds toolchain (protocol 2) as a Mix task. A thin shim over the bundled instruments (Python, stdlib); never a port.

Current section

Files

Jump to
dds priv toolchain local_judge.py
Raw

priv/toolchain/local_judge.py

#!/usr/bin/env python3
"""local_judge.py — the judged leg on Claude Code PLAN tokens.
Drives one headless `claude -p` session to judge the nominees, instead
of the Message Batches API: no ANTHROPIC_API_KEY, no API billing —
the user's Claude Code subscription does the judging. The trust
boundary is unchanged and indifferent to the runtime: the session
writes .dds-verdicts.json and the gate quote-verifies and REPLAYS
every record exactly as it does for batch or interactive verdicts.
The session is told to self-verify: run the instrument with
--verdicts itself and repair rejections before finishing — the
interactive agent leg's re-run-until-clean loop, in one headless run.
Refuses unprofiled platforms (extraction is not a judging problem),
mirroring batch_judge.
"""
import json
import os
import shutil
import subprocess
import sys
HERE = os.path.dirname(os.path.abspath(__file__))
PROMPT = """Judge the nominated ADRs of this repository's dds check and write the
verdicts file. You are the judged leg of /dds-check: narrow per-ADR
conformance verdicts, produced by investigation, enforced externally —
your record is verified by an instrument you do not control.
## Procedure
1. Run the instrument for the nominee list and witnessed facts:
{py} {instrument} --repo . --base {base} --json /tmp/dds-pre.json
Read the `nominees` from /tmp/dds-pre.json (paths of ADRs to judge)
and use the walk for coverage/context. Read each nominated ADR.
2. Investigate with structural retrieval nets:
{py} {tsq} call <name> --repo . [--in lib/]
A query is a NET defining the evaluated population; YOU evaluate the
retrieved source (read files freely for context). Prefer broad nets
and filter visibly in your basis. Constructed tree-sitter queries:
write the query to a file and run `{py} {tsq} --scm <file> --repo .`.
3. Write the verdicts to {out} — a BARE JSON list, one record per
nominee:
{{"adr": "docs/adr/....md",
"verdict": "conformant|partial|unwitnessed|not_verifiable",
"rule_quote": "<the load-bearing sentence, VERBATIM from the ADR's ## Rule section>",
"basis": "your evaluation, incl. visible filtering of matches",
"evidence": [{{"query": "call <name>", // or "query_scm": "<raw query>"
"in": "lib/", "expect": "some"|"none",
"matches": [{{"at": "file:line", "text": "<span EXACTLY as tsq printed>"}}],
"controls": [{{"source": "<code the query provably matches>", "matches": 1}}], // absence-via-scm only
"note": "what this operationalizes"}}],
"dormant_mechanism": "<optional>"}}
4. SELF-VERIFY, then finish: re-run
{py} {instrument} --repo . --base {base} --verdicts {out}
If it reports rejected verdicts (DEGRADED lines naming them) or
missing nominees, fix the records and re-run — at most 3 repair
rounds; then stop and leave the honest state.
## The law (violations are rejected by the instrument)
- rule_quote: verbatim substring of the ADR's ## Rule section — never
the preamble, never paraphrase.
- matches: copied EXACTLY from tsq output, never from memory or a file
read. Presence evidence must reproduce on replay.
- absence via `call`: citable only when tsq reports 0 structural AND
0 textual hits (tsq warns otherwise). Absence via a constructed
query: REQUIRES positive controls (code the query provably matches).
- conformant needs POSITIVELY shown state; an interim rule accurately
declaring the witnessed state is conformant AS an interim. A rule
that resists operationalization is honestly not_verifiable, naming
the searches run. Judge witnessed state, never your recollection of
how such apps usually work.
- The verdicts file is the deliverable — no report, no commentary,
never commit anything.
"""
def run(repo, base, out, say):
claude = shutil.which("claude")
if not claude:
return False
instrument = os.path.join(HERE, "check_join.py")
tsq = os.path.join(HERE, "tsq.py")
prompt = PROMPT.format(
py=sys.executable, instrument=instrument, tsq=tsq, base=base, out=out
)
say(f"judging via local claude (plan tokens) — headless session, self-verifying…")
r = subprocess.run(
[
claude, "-p", prompt,
"--output-format", "json",
"--max-turns", "60",
"--allowedTools", "Bash,Read,Write,Edit,Glob,Grep",
],
cwd=repo, capture_output=True, text=True, timeout=1800,
)
if r.returncode != 0:
say(f"local claude exited {r.returncode}: {(r.stderr or r.stdout or '').strip()[:200]}")
return os.path.exists(out)
try:
doc = json.loads(r.stdout)
cost = doc.get("total_cost_usd")
# claude -p reports computed usage regardless of billing mode;
# on plan/subscription auth this is covered usage, not a bill
say(
f"local judge done — {doc.get('num_turns', '?')} turns, "
f"{round((doc.get('duration_ms') or 0) / 1000)}s"
+ (f", usage ≈ ${cost:.2f} (plan-covered)" if cost else "")
)
except (json.JSONDecodeError, TypeError):
say("local judge done (unparseable session summary)")
return os.path.exists(out)