ADR 0021: Task-brief quality score

Field	Value
Status	Accepted
Date	2026-06-01
Deciders	Project maintainers
Supersedes	—
Superseded by	—
Related ADRs	—

Summary

Add a deterministic task-brief quality rubric to @aidokit/core — scoreTaskBrief(text) — returning { score, grade, breakdown }. A new advisory CLI command aidokit task score <id|path> resolves a brief (the same way aidokit eval does) and prints the score. The command always exits 0; it informs, it does not gate.

Context

A common failure mode (ROI gap analysis, epic #10) is under-specified task briefs entering the build loop: no acceptance criteria, no rollback, no test plan. The orchestrator and humans benefit from a fast, deterministic signal on brief completeness before work starts. aidokit makes zero LLM calls, so the rubric must be a pure, keyword-driven structural check — not a semantic judgement.

Decision

Rubric (total 100)

Dimension	Max	Met when
scope / objective	20	H2/H3 heading containing `scope`, `objective`, or `goal`
acceptance criteria	30	`acceptance criteria` heading and ≥1 runnable `>`-prefixed checkbox
risks	15	heading containing `risk`
rollback / back-out	15	heading containing `rollback`, `back-out`, `backout`, or `back out`
test plan / validation commands	20	heading containing `test plan`, `validation`, etc.

Each dimension is all-or-nothing. Grades: A ≥ 90, B ≥ 75, C ≥ 60, D ≥ 40, F < 40.

The runnable-checkbox requirement reuses the aidokit eval convention: a - [ ] > <cmd> checkbox is machine-runnable, so a brief that ships at least one is materially more verifiable than prose-only criteria.

API

scoreTaskBrief(text: string): {
  score: number;
  grade: 'A' | 'B' | 'C' | 'D' | 'F';
  breakdown: { dimension: string; points: number; max: number; met: boolean }[];
}

CLI

aidokit task score [id|path] [--brief <path>] [--json]. Brief resolution mirrors aidokit eval: an explicit --brief/path argument wins; otherwise a bare id resolves to agent-artifacts/task-briefs/<id>.md. Advisory — exit 0 always.

Consequences

Positive

Deterministic, zero-LLM, instant feedback on brief completeness.
Reuses the existing brief-resolution and runnable-checkbox conventions.
Advisory exit code means it can be wired into the orchestrator without blocking work on a low score.

Negative

Keyword-driven; a well-structured brief using unusual headings could score low. Synonym lists mitigate the common cases; the rubric is intentionally shallow (YAGNI) and can grow via a non-breaking edit.

Neutral

The rubric weights are a v1.0 starting point, encoded as code, not config.

Alternatives considered

Gate the build loop on score. Rejected: a structural heuristic should not block work. Advisory only.
Configurable rubric. Rejected as premature (YAGNI). Weights live in code.

Validation

[ ] scoreTaskBrief returns 100/A for a brief with all five sections and a runnable checkbox; 0/F for empty.
[ ] Acceptance-criteria dimension is unmet without a >-prefixed checkbox.
[ ] aidokit task score <id> resolves and scores a brief and exits 0.