Conformance Levels Specification

Formal specification of the three conformance levels (Minimum / Standard / Strict), the per-level checklists each level requires, and the verification harness that enforces them.

Field Value
Spec name aidokit Conformance Levels
Spec version 1.0 (draft)
Status Pre-alpha — checklists frozen at spec freeze; harness implementation evolves
Implements @aidokit/core conformance harness
Sibling specs adapter-contract.md, stack-pack-contract.md
Last reviewed Pre-publication

Table of contents

  1. Purpose and audience
  2. Conventions
  3. Conformance model
  4. Adapter conformance — per-level checklists
  5. Stack-pack conformance — per-level checklists
  6. The conformance harness
  7. Test fixtures
  8. Reporting format
  9. CI integration
  10. aidokit doctor integration
  11. Project-level conformance
  12. Trust signals and marketplace
  13. Failure modes and recovery
  14. Versioning and evolution
  15. Open questions
  16. Appendix A: Synthetic ProjectContext fixtures
  17. Appendix B: Full check id reference

1. Purpose and audience

1.1 Purpose

This specification defines:

adapter-contract.md §11 and stack-pack-contract.md §15 declare which methods are MUST/SHOULD/MAY at each level. This spec aggregates those declarations into executable checklists that the harness implements. If there is a conflict, the contract specs win for definitions; this spec wins for verification details.

1.2 Audience

1.3 Non-purpose

This document does NOT:


2. Conventions

2.1 RFC 2119 keywords

The key words MUST, MUST NOT, SHOULD, SHOULD NOT, MAY, and REQUIRED are interpreted as described in RFC 2119.

2.2 Check identifiers

Every check has a stable identifier of the form <target>.<category>.<short-name>:

adapter.emit-rules.exactly-one-file
adapter.emit-verbs.intake-present
adapter.watchdog.test-loop-detection
stack-pack.detect.returns-result
stack-pack.skills.has-review-markers

Identifiers MUST NOT be renamed once published. Identifiers MAY be added in minor spec versions.

2.3 Checklist notation

Each check in this spec uses:

□ <check-id> — <description>
  Method/area: <what's tested>
  Required: MUST | SHOULD
  Pass when: <pass criterion>
  Fail when: <fail criterion>

3. Conformance model

3.1 Levels are capability outcomes, not a ranking

A conformance level describes what a target CLI can physically support, not a grade of adapter quality. An adapter declares the highest level its target CLI can honestly back: the level is a consequence of the platform's primitives, not a score one adapter "wins" over another. aidokit's three first-party adapters are equal siblings (ADR-0016, ADR-0031); they differ in declared level only because their target CLIs differ in capability, not in care or completeness.

Level Capability the target CLI must expose
Minimum Targets implementing only the MUST requirements from the relevant contract
Standard Minimum + all SHOULD requirements
Strict Standard + all MAY requirements promoted to expected

3.2 Per-adapter capability matrix

The level each first-party adapter declares follows directly from whether its target CLI exposes deterministic hooks (PreToolUse / PostToolUse) and output styles. Where those primitives are absent, the equivalent guidance ships at the prompt level instead — which is honest, not lesser. Adapters are listed alphabetically.

Adapter Hooks Output styles Declared level Why
codex No No (prompt-level only) Minimum Codex CLI exposes neither hooks nor output styles, so watchdog/styling are prompt-level
copilot No No Minimum GitHub Copilot CLI exposes neither hooks nor output styles
claude-code Yes Yes Strict Claude Code exposes deterministic hooks and output styles, enabling enforced guarantees

This asymmetry is honest and intended to remain: it reflects target-CLI capability, not a quality difference between adapters. The per-level checklists in §4 below are the precise expression of these capability outcomes.

3.3 Declaration vs verification

Every target declares a conformance level in its manifest (AdapterManifest.conformance or StackPackManifest.conformance). The harness verifies the claim by running the level's checks against the target's implementation.

Declared > Verified is a conformance violation. The package CI MUST fail. Declared < Verified is acceptable (over-conforming). The harness reports the actual level.

3.4 Inheritance

A pass at level X means the target passed every check at every level ≤ X.

3.5 Per-method vs holistic

Checks fall into two categories:

Both run in the same harness pass.


4. Adapter conformance — per-level checklists

4.1 Minimum conformance — adapter

These checks MUST all pass for an adapter to claim Minimum conformance.

4.1.1 Manifest checks

□ adapter.manifest.declared-level
  Method/area: manifest
  Required: MUST
  Pass when: manifest.conformance is one of 'minimum' | 'standard' | 'strict'
  Fail when: missing or invalid value

□ adapter.manifest.spec-version
  Method/area: manifest
  Required: MUST
  Pass when: manifest.specVersion === '2.0'
  Fail when: any other value or missing

□ adapter.manifest.capabilities-declared
  Method/area: manifest.capabilityDeclarations
  Required: MUST
  Pass when: writesPaths and runsShellCommands fields are present arrays
  Fail when: missing or non-array

□ adapter.manifest.cli-target
  Method/area: manifest
  Required: MUST
  Pass when: cliTarget is a non-empty string with version spec
  Fail when: empty or missing

4.1.2 Agent rules file checks

□ adapter.emit-rules.exactly-one-file
  Method/area: emitAgentRulesFile
  Required: MUST
  Pass when: returns a single EmittedFile object (not array, not null)
  Fail when: returns anything else

□ adapter.emit-rules.owner-is-adapter
  Method/area: emitAgentRulesFile
  Required: MUST
  Pass when: returned file has owner: 'adapter'
  Fail when: any other owner

□ adapter.emit-rules.references-source-of-truth-chain
  Method/area: emitAgentRulesFile
  Required: MUST
  Pass when: file content includes the 10-level source-of-truth chain
  Fail when: chain missing or incomplete

□ adapter.emit-rules.includes-validation-commands
  Method/area: emitAgentRulesFile
  Required: MUST
  Pass when: file content references at least one validation command
  Fail when: no validation commands referenced

4.1.3 Engine config checks

□ adapter.emit-engine.paths-under-engine-dir
  Method/area: emitEngineConfig
  Required: MUST
  Pass when: every returned EmittedFile.path is under the adapter's declared engine directory
  Fail when: any file outside the engine directory

□ adapter.emit-engine.all-paths-relative
  Method/area: emitEngineConfig
  Required: MUST
  Pass when: every path is relative (no leading /, no '..' segments)
  Fail when: any absolute path or upward traversal

4.1.4 Verb checks

□ adapter.emit-verbs.intake-present
  Method/area: emitVerbs
  Required: MUST
  Pass when: output contains a file implementing the 'intake' verb
  Fail when: missing intake implementation

□ adapter.emit-verbs.implement-task-present
  Method/area: emitVerbs
  Required: MUST
  Pass when: output contains a file implementing the 'implement-task' verb
  Fail when: missing

□ adapter.emit-verbs.orchestrator-next-present
  Method/area: emitVerbs
  Required: MUST
  Pass when: output contains a file implementing the 'orchestrator-next' verb
  Fail when: missing

□ adapter.emit-verbs.intake-stops-before-code
  Method/area: emitVerbs (intake)
  Required: MUST
  Pass when: intake verb content references the hard-stop-before-code requirement
  Fail when: no stop directive present

4.1.5 Role checks

□ adapter.emit-roles.six-required-roles-present
  Method/area: emitRoles
  Required: MUST
  Pass when: output contains role files for: researcher, architect, planner,
             builder, tester-reviewer, maintainer
  Fail when: any of the six is missing

□ adapter.emit-roles.builder-scope-limited
  Method/area: emitRoles (builder)
  Required: MUST
  Pass when: builder role declares scope-limited-write capability profile
  Fail when: builder runs with broader or narrower profile

□ adapter.emit-roles.read-only-roles-correct
  Method/area: emitRoles
  Required: MUST
  Pass when: researcher, architect, planner, tester-reviewer declare read-only profile
  Fail when: any of those roles can write

□ adapter.emit-roles.seventh-role
  Method/area: emitRoles (frontend-browser-tester)
  Required: MAY (Minimum), SHOULD (Standard), MUST (Strict)
  Pass when: at Strict, frontend-browser-tester role file is present
  Skip when: at Minimum and role is absent

4.1.6 Skill checks

□ adapter.emit-skills.required-skills-emitted
  Method/area: emitSkills
  Required: MUST
  Pass when: every skill in input with required:true appears in output
  Fail when: any required skill missing

□ adapter.emit-skills.preserves-triggers
  Method/area: emitSkills
  Required: MUST
  Pass when: emitted skills preserve autoLoadTriggers and preloadedBy from input
  Fail when: metadata lost

□ adapter.emit-skills.stack-pack-skills-marked
  Method/area: emitSkills
  Required: SHOULD
  Pass when: stack-pack-contributed skills are visibly marked (header or path convention)
  Warn when: indistinguishable from core skills

4.1.7 Watchdog checks

□ adapter.watchdog.test-loop-detection
  Method/area: emitWatchdog
  Required: MUST
  Pass when: watchdog includes mechanism to stop after 3 consecutive test failures
  Fail when: no such mechanism

□ adapter.watchdog.command-loop-detection
  Method/area: emitWatchdog
  Required: MUST
  Pass when: watchdog includes mechanism to stop after 5 same-command failures
  Fail when: no such mechanism

□ adapter.watchdog.file-edit-counter
  Method/area: emitWatchdog
  Required: MUST
  Pass when: watchdog tracks per-task file-edit counts and warns at threshold
  Fail when: no edit tracking

□ adapter.watchdog.repair-cap
  Method/area: emitWatchdog
  Required: MUST
  Pass when: watchdog enforces max 2 repair attempts per task
  Fail when: no cap

□ adapter.watchdog.out-of-scope-block
  Method/area: emitWatchdog
  Required: MUST
  Pass when: watchdog blocks writes to files outside the active task's in-scope list
  Fail when: not enforced

□ adapter.watchdog.version-bump-gate
  Method/area: emitWatchdog
  Required: MUST
  Pass when: watchdog blocks version bumps without explicit approval flag
  Fail when: not enforced

□ adapter.watchdog.tracked-deletion-gate
  Method/area: emitWatchdog
  Required: MUST
  Pass when: watchdog blocks deletion of tracked files without approval
  Fail when: not enforced

□ adapter.watchdog.blocker-emitter
  Method/area: emitWatchdog
  Required: MUST
  Pass when: a watchdog stop emits a blocker report under agent-artifacts/blockers/
  Fail when: no blocker emission

□ adapter.watchdog.monitoring-stream
  Method/area: emitWatchdog
  Required: SHOULD (Standard), MUST (Strict)
  Pass when: hook writes events.jsonl under agent-artifacts/monitoring/
  Skip when: at Minimum and feature absent

4.1.8 Post-install checks

□ adapter.post-install.executable-scripts-marked
  Method/area: emitWatchdog / postInstall
  Required: MUST
  Pass when: every emitted .mjs / .sh file is marked mode: 0o755 OR a chmod command is in postInstall
  Fail when: scripts emitted without execute permissions

□ adapter.post-install.beads-init-when-enabled
  Method/area: postInstall
  Required: SHOULD
  Pass when: if ctx.beadsEnabled, postInstall includes a bd init command
  Warn when: Beads enabled but not initialized

4.1.9 Determinism checks

□ adapter.determinism.repeated-invocation-stable
  Method/area: all emit methods
  Required: MUST
  Pass when: invoking the same method twice with identical ctx produces byte-identical output
  Fail when: outputs differ

4.2 Standard conformance — adapter

Adds to Minimum:

□ adapter.mcp.install-implemented
  Method/area: installMCP
  Required: SHOULD (Minimum), MUST (Standard)
  Pass when: returns valid ShellCommand[] for the test MCP fixture
  Fail when: throws or returns invalid commands

□ adapter.mcp.remove-implemented
  Method/area: removeMCP
  Required: SHOULD (Minimum), MUST (Standard)
  Pass when: returns valid removal commands; idempotent (no error if MCP absent)
  Fail when: throws or fails on absent MCP

□ adapter.mcp.list-implemented
  Method/area: listInstalledMCPs
  Required: SHOULD (Minimum), MUST (Standard)
  Pass when: returns an array (possibly empty)
  Fail when: throws or returns non-array

□ adapter.sync.pre-sync-implemented
  Method/area: preSync
  Required: SHOULD (Minimum), MUST (Standard)
  Pass when: returns a valid SyncPlan
  Fail when: throws or returns malformed plan

□ adapter.sync.classifies-all-files
  Method/area: preSync
  Required: MUST (Standard)
  Pass when: every file in the would-emit set appears in exactly one of toCreate/toUpdate/toDelete/toSkip
  Fail when: any file unclassified or duplicated

□ adapter.doctor.implemented
  Method/area: doctor
  Required: SHOULD (Minimum), MUST (Standard)
  Pass when: returns DoctorCheck[]; runs all checks (no short-circuit)
  Fail when: throws or short-circuits on first failure

□ adapter.doctor.verifies-cli-installation
  Method/area: doctor
  Required: MUST (Standard)
  Pass when: doctor checks target CLI is installed and version matches manifest.cliTarget
  Fail when: absent

□ adapter.doctor.verifies-required-files
  Method/area: doctor
  Required: MUST (Standard)
  Pass when: doctor reports on presence of expected engine directory contents
  Fail when: absent

4.3 Strict conformance — adapter

Adds to Standard:

□ adapter.emit-roles.seventh-role-strict
  Method/area: emitRoles
  Required: MUST (Strict)
  Pass when: frontend-browser-tester role is emitted
  Fail when: missing

□ adapter.output-styles.emitted
  Method/area: emitOutputStyles
  Required: SHOULD (Standard), MUST (Strict)
  Pass when: returns at least one output-style file
  Fail when: returns empty array at Strict

□ adapter.watchdog.monitoring-stream-strict
  Method/area: emitWatchdog
  Required: MUST (Strict)
  Pass when: monitoring stream writes structured events for every workflow stage transition
  Fail when: incomplete coverage

□ adapter.doctor.runs-conformance-subset
  Method/area: doctor
  Required: SHOULD (Strict)
  Pass when: doctor invokes a subset of the conformance harness
  Warn when: absent

4.4 Adapter conformance summary table

Level Total MUST checks Total SHOULD checks
Minimum ~24 ~3
Standard ~32 ~6
Strict ~36 ~8

Exact counts depend on spec evolution; the harness is authoritative.


5. Stack-pack conformance — per-level checklists

5.1 Minimum conformance — stack pack

5.1.1 Manifest checks

□ stack-pack.manifest.declared-level
  Required: MUST
  Pass when: manifest.conformance is valid

□ stack-pack.manifest.languages-non-empty
  Required: MUST
  Pass when: manifest.languages is a non-empty array

□ stack-pack.manifest.spec-version
  Required: MUST
  Pass when: manifest.specVersion === '2.0'

5.1.2 Detection checks

□ stack-pack.detect.returns-result
  Method/area: detect
  Required: MUST
  Pass when: returns a DetectionResult on every invocation
  Fail when: throws

□ stack-pack.detect.confidence-categorical
  Method/area: detect
  Required: MUST
  Pass when: result.confidence is one of 'high' | 'medium' | 'low' | 'none'
  Fail when: any other value

□ stack-pack.detect.no-shell-calls
  Method/area: detect
  Required: MUST
  Pass when: detection uses only DetectContext methods (verified via harness sandbox)
  Fail when: shell execution detected

□ stack-pack.detect.handles-missing-files
  Method/area: detect
  Required: MUST
  Pass when: returns matched:false on a project with no relevant signals
  Fail when: throws on absent files

5.1.3 Skill checks

□ stack-pack.skills.at-least-one-skill
  Method/area: suggestSkills
  Required: MUST
  Pass when: returns at least one SkillTemplate
  Fail when: returns empty array

□ stack-pack.skills.source-marked
  Method/area: suggestSkills
  Required: MUST
  Pass when: every skill has source: 'stack-pack' and sourcePackId set
  Fail when: any skill missing these fields

□ stack-pack.skills.has-review-markers
  Method/area: suggestSkills
  Required: MUST
  Pass when: opinionated skills contain <REVIEW REQUIRED> markers (heuristic check)
  Fail when: opinionated language without markers detected

□ stack-pack.skills.no-secrets
  Method/area: suggestSkills
  Required: MUST
  Pass when: skill content contains no values matching common secret patterns
  Fail when: API key / token / password patterns detected

□ stack-pack.skills.no-hard-coded-paths
  Method/area: suggestSkills
  Required: SHOULD
  Pass when: skill content does not contain absolute paths or user-home references
  Warn when: detected

5.1.4 Determinism checks

□ stack-pack.determinism.repeated-invocation-stable
  Method/area: all methods
  Required: MUST
  Pass when: repeated invocation with identical ctx produces identical output
  Fail when: outputs differ

5.2 Standard conformance — stack pack

Adds to Minimum:

□ stack-pack.mcp.references-catalog-ids
  Method/area: suggestMCPs
  Required: MUST (Standard)
  Pass when: every returned id exists in the current MCP catalog
  Fail when: any id not in catalog

□ stack-pack.mcp.no-sensitive-suggestions
  Method/area: suggestMCPs
  Required: MUST
  Pass when: no returned id has securitySensitive: true in the catalog
  Fail when: any sensitive id returned

□ stack-pack.validation.commands-well-formed
  Method/area: suggestValidationCommands
  Required: MUST (Standard)
  Pass when: every returned command has valid id, label, command, kind
  Fail when: missing fields

□ stack-pack.validation.ci-safety-marked
  Method/area: suggestValidationCommands
  Required: MUST
  Pass when: ciSafe field is set on every command
  Fail when: missing

□ stack-pack.validation.respects-project-scripts
  Method/area: suggestValidationCommands
  Required: SHOULD
  Pass when: when test fixture has a package.json with matching scripts, commands use 'npm run <script>'
  Warn when: pack reinvents commands that exist as scripts

5.3 Strict conformance — stack pack

Adds to Standard:

□ stack-pack.architecture.pattern-implemented
  Method/area: defaultArchitecturePattern
  Required: SHOULD (Strict)
  Pass when: method exists and returns a valid ArchitecturePatternHint
  Skip when: method is absent (allowed by spec)

□ stack-pack.architecture.suggested-layout-non-empty
  Method/area: defaultArchitecturePattern
  Required: MUST when method exists
  Pass when: suggestedLayout has at least one entry
  Fail when: empty when method is implemented

□ stack-pack.composability.composes-with-declared
  Method/area: manifest.composesWith
  Required: SHOULD (Strict)
  Pass when: manifest declares known composable packs (informational)
  Warn when: missing for packs that commonly compose

5.4 Stack pack conformance summary table

Level Total MUST checks Total SHOULD checks
Minimum ~11 ~2
Standard ~16 ~3
Strict ~18 ~4

6. The conformance harness

6.1 Public API

Exported from @aidokit/core:

import type { Adapter, StackPack, ConformanceLevel, ConformanceReport } from '@aidokit/core';

export async function runAdapterConformance(
  adapter: Adapter,
  level: ConformanceLevel,
  options?: ConformanceOptions,
): Promise<ConformanceReport>;

export async function runStackPackConformance(
  pack: StackPack,
  level: ConformanceLevel,
  options?: ConformanceOptions,
): Promise<ConformanceReport>;

export interface ConformanceOptions {
  /** Override fixture for testing. */
  fixture?: ProjectContext;

  /** Subset of check ids to run (default: all at the level). */
  onlyChecks?: string[];

  /** Skip these check ids. */
  skipChecks?: string[];

  /** Treat SHOULD failures as warnings (default) or failures. */
  strictSoft?: boolean;
}

6.2 Harness flow

1. Identify target type (adapter | stack-pack)
2. Resolve check set for declared level
3. Filter by onlyChecks / skipChecks
4. Build synthetic ProjectContext fixtures (or use override)
5. For each check:
   a. Invoke the target method(s) the check covers
   b. Apply pass/fail criterion
   c. Record ConformanceResult with evidence
6. Aggregate into ConformanceReport
7. Return

6.3 Failure isolation

A check's failure MUST NOT prevent subsequent checks from running. The harness collects all results before returning.

6.4 Performance budget

A full conformance run (~36 adapter checks or ~18 stack-pack checks) MUST complete in under 30 seconds on a developer machine, exclusive of any external CLI invocations.


7. Test fixtures

7.1 Why fixtures

The harness needs synthetic ProjectContext (or DetectContext) objects that exercise different scenarios without requiring real projects. Fixtures are deterministic, version-controlled, and shared across all targets.

7.2 Standard fixtures

Each fixture is a named ProjectContext instance in @aidokit/core/fixtures:

Fixture id Description
greenfield-node-ts New Node.js + TypeScript project, no source code yet
brownfield-node-ts Existing Node-TS project with README and prior PRD
multi-stack-fullstack Node-TS + React + Postgres composition
python-django Python + Django brownfield
go-cli Go CLI project
monorepo-mixed Monorepo with TS frontend + Python backend
minimum-conformance Project targeting minimum conformance
strict-conformance Project targeting strict conformance

Each fixture has a corresponding DetectContext view derived from its file tree.

7.3 Fixture extensibility

Targets MAY contribute additional fixtures via the ConformanceOptions.fixture override. The standard fixtures cover the v1.0 spec; new fixtures appear with new spec versions.

7.4 Determinism

Fixtures MUST be deterministic: re-running the same harness against the same target with the same fixture produces identical results. No network access; no clock-dependent fields; no random data.


8. Reporting format

8.1 ConformanceResult

export interface ConformanceResult {
  /** Stable check identifier (see §2.2). */
  id: string;

  /** Human-readable description of what was tested. */
  description: string;

  /** Which level this check applies to. */
  level: ConformanceLevel;

  /** MUST | SHOULD | MAY. */
  required: 'MUST' | 'SHOULD' | 'MAY';

  /** Outcome. */
  status: 'pass' | 'fail' | 'skip' | 'warn';

  /** Detail message. */
  message: string;

  /** Evidence for failures. */
  evidence?: {
    method?: string;
    expected?: unknown;
    actual?: unknown;
    fixture?: string;
  };

  /** Suggested fix (optional). */
  remediation?: string;

  /** Spec reference (filename + section). */
  specRef?: string;
}

8.2 ConformanceReport

export interface ConformanceReport {
  /** Whether this was an adapter or stack pack run. */
  target: 'adapter' | 'stack-pack';

  /** Name of the target (manifest.name). */
  targetName: string;

  /** Target's declared conformance level. */
  declaredLevel: ConformanceLevel;

  /** Level the harness verified against. */
  testedLevel: ConformanceLevel;

  /** All check results, in execution order. */
  results: ConformanceResult[];

  /** Summary counts. */
  summary: {
    total: number;
    passed: number;
    failed: number;
    skipped: number;
    warned: number;
  };

  /** Overall pass/fail at testedLevel. */
  overall: 'pass' | 'fail';

  /** Timestamp of the run. */
  ranAt: string;

  /** Versions of relevant components. */
  versions: {
    aidoCore: string;
    specVersion: string;
    targetVersion: string;
  };
}

8.3 Human-readable formatter

The harness ships a default formatter:

Conformance Report: claude-code@0.1.0
Declared: Strict · Tested: Strict
Spec: v2.0 · Harness: @aidokit/core@1.2.0

Summary: 32 passed · 0 failed · 4 skipped · 2 warned
Overall: PASS

Skipped:
  · adapter.emit-roles.seventh-role-strict — only required at Strict
  · adapter.output-styles.emitted — only required at Strict
  · (2 more)

Warnings:
  ⚠ adapter.post-install.beads-init-when-enabled
    Beads is enabled in fixture but no `bd init` in postInstall.
    Remediation: add bd init to postInstall when ctx.beadsEnabled.

8.4 Machine-readable formatter

--format=json (when invoked via CLI) emits the ConformanceReport object directly.


9. CI integration

9.1 Adapter package CI

Every adapter package MUST include a test:conformance script:

{
  "scripts": {
    "test:conformance": "vitest run conformance.test.ts"
  }
}

And a conformance test file (packages/adapter-*/test/conformance.test.ts):

import { test, expect } from 'vitest';
import { runAdapterConformance } from '@aidokit/core';
import { adapter } from '../src/index.js';

test(`${adapter.manifest.name} conforms to declared level`, async () => {
  const report = await runAdapterConformance(adapter, adapter.manifest.conformance);
  if (report.overall !== 'pass') {
    const failures = report.results.filter((r) => r.status === 'fail');
    console.error(JSON.stringify(failures, null, 2));
  }
  expect(report.overall).toBe('pass');
});

9.2 Stack pack package CI

Mirror the above for stack packs:

import { runStackPackConformance } from '@aidokit/core';
import { stackPack } from '../src/index.js';

test(`${stackPack.manifest.name} conforms to declared level`, async () => {
  const report = await runStackPackConformance(stackPack, stackPack.manifest.conformance);
  expect(report.overall).toBe('pass');
});

9.3 Required gates

The package's CI workflow MUST run pnpm test:conformance and fail the build on non-zero exit. Publishing a package that fails conformance is prohibited.

9.4 Reproducibility

Conformance reports MUST be reproducible: a given target version + harness version + spec version always produces the same report (modulo timestamps).


10. aidokit doctor integration

10.1 Subset run

aidokit doctor re-runs a subset of the harness on the user's installed adapter to detect:

The subset (~10–12 checks) is fast (<5 seconds) and focuses on structural integrity.

10.2 Drift detection

aidokit doctor compares the live conformance report to the report captured at install time (stored in .aido/conformance-snapshot.json). Differences are reported.

10.3 Exit semantics

10.4 What aidokit doctor doesn't do

aidokit doctor does NOT:

For a full conformance check, run the package CI in the adapter's repo.


11. Project-level conformance

11.1 Declaration

A user's project declares its target conformance level in <projectRoot>/.aido/adapter.md:

---
adapter: claude-code
conformanceTarget: standard
specVersion: '2.0'
---

11.2 Constraint

The project's conformanceTarget MUST be <= the installed adapter's manifest.conformance. A project at Strict requires an adapter declaring Strict.

aidokit init enforces this at install time:

✘ Adapter 'codex' declares Minimum conformance; cannot target Strict.
  Either choose a different adapter or lower your target.

11.3 Inheritance

The project conformance level applies to every operation: file emission verification, doctor checks, MCP suggestion strictness.


12. Trust signals and marketplace

12.1 Verified badge

The post-1.0 marketplace (v2.0 per ROADMAP.md) grants a "Verified" badge to packages that:

  1. Pass conformance in CI on every published version
  2. Make conformance reports publicly accessible (e.g., in repo as conformance-report.json)
  3. Sign packages with npm provenance

12.2 Trust tiers

Tier Criteria
Official @aidokit/* scope; maintained by core team
Verified Passes conformance CI; signed; active maintainer
Community Published; follows naming convention; no other signals
Unverified Matches naming but has known issues (deprecated, abandoned, etc.)
$ aido search adapter

@aidokit/adapter-codex            official    @ Minimum conformance (verified)
@aidokit/adapter-copilot          official    @ Minimum conformance (verified)
@aidokit/adapter-claude-code      official    @ Strict conformance (verified)
aidokit-adapter-aider             community   @ claimed Standard (unverified)
@acme/aidokit-adapter-mytool      verified    @ Minimum conformance

13. Failure modes and recovery

13.1 Failure during aidokit init

If aidokit init invokes a conformance subset on the adapter being installed and it fails:

✘ Adapter conformance check failed: claude-code@0.1.0 (Strict claimed)
  Failing checks:
    · adapter.emit-roles.builder-scope-limited
      Builder role declared default permissionMode, expected with PreToolUse hook
  Try:  pnpm update @aidokit/adapter-claude-code

Exit code: 13 (CONFORMANCE_FAILED).

13.2 Failure during aidokit doctor

✘ Conformance drift detected
  · adapter.emit-skills.required-skills-emitted
    Skill 'beads-queries' missing from .claude/skills/

  Try:  aido sync

13.3 Recovery strategies

Failure Recovery
Adapter version mismatch aidokit sync to re-emit
Skill file deleted aidokit sync --skills
Hook scripts lost execute permission aidokit sync --scripts or chmod +x manually
Adapter no longer claims declared level aidokit doctor --strict to inspect; upgrade or replace adapter
MCP scoping divergence aido mcp scope <id> --set-roles <list>

14. Versioning and evolution

14.1 Check stability

Once published, a check id is stable. The pass/fail criterion MAY be refined (with care) but the id and its semantic intent MUST NOT change.

14.2 Adding checks

New checks may be added in minor spec versions. Existing targets retain their declared conformance (the new check applies to the next major spec version).

14.3 Removing checks

A check MAY be marked deprecated (excluded from current runs) but MUST remain in the spec for historical reference.

14.4 Level redefinition

Promoting a check from SHOULD to MUST is a breaking change (major bump). Demoting is non-breaking.


15. Open questions

Question Resolution target
Should --strict doctor mode be the default in CI environments? Spec 1.0 freeze
Whether to add a Conformance ID badge SVG generator v2.0 marketplace
How to handle conformance for adapters that target older spec versions Spec 1.1
Whether to ship a aidokit conformance-report standalone CLI command v1.0
Whether targets should declare which checks they intentionally skip (with rationale) Spec 1.1

Appendix A: Synthetic ProjectContext fixtures

A.1 greenfield-node-ts

{
  projectRoot: '/tmp/aidokit-conformance/greenfield-node-ts',
  projectName: 'example-app',
  stack: {
    languages: ['typescript'],
    frameworks: [],
  },
  brownfield: false,
  sourceDocs: [],
  adapterName: '<provided-by-harness>',
  conformance: 'standard',
  beadsEnabled: true,
  selectedStackPacks: ['node-ts'],
  selectedMCPs: ['context7', 'beads-mcp'],
  os: 'linux',
  arch: 'x64',
  installedPrereqs: [
    { id: 'node',  version: '20.11.0', installed: true },
    { id: 'git',   version: '2.43.0',  installed: true },
    { id: 'beads', version: '0.9.0',   installed: true },
  ],
  aidoCliVersion: '0.1.0',
  specVersion: '2.0',
  options: {},
}

A.2 brownfield-node-ts

Same as greenfield-node-ts but with:

{
  brownfield: true,
  sourceDocs: [
    { path: 'docs/inputs/source-prd.md', kind: 'prd' },
    { path: 'README.md',                  kind: 'readme' },
  ],
}

A.3 multi-stack-fullstack

{
  // ... same base
  selectedStackPacks: ['node-ts', 'react', 'postgres'],
  selectedMCPs: ['context7', 'beads-mcp', 'playwright', 'postgres-mcp'],
}

A.4 strict-conformance

{
  // ... base
  conformance: 'strict',
}

Appendix B: Full check id reference

Generated from this spec; canonical list lives in @aidokit/core/src/conformance/checks.ts.

B.1 Adapter checks (~36)

adapter.manifest.declared-level
adapter.manifest.spec-version
adapter.manifest.capabilities-declared
adapter.manifest.cli-target
adapter.emit-rules.exactly-one-file
adapter.emit-rules.owner-is-adapter
adapter.emit-rules.references-source-of-truth-chain
adapter.emit-rules.includes-validation-commands
adapter.emit-engine.paths-under-engine-dir
adapter.emit-engine.all-paths-relative
adapter.emit-verbs.intake-present
adapter.emit-verbs.implement-task-present
adapter.emit-verbs.orchestrator-next-present
adapter.emit-verbs.intake-stops-before-code
adapter.emit-roles.six-required-roles-present
adapter.emit-roles.builder-scope-limited
adapter.emit-roles.read-only-roles-correct
adapter.emit-roles.seventh-role
adapter.emit-roles.seventh-role-strict
adapter.emit-skills.required-skills-emitted
adapter.emit-skills.preserves-triggers
adapter.emit-skills.stack-pack-skills-marked
adapter.watchdog.test-loop-detection
adapter.watchdog.command-loop-detection
adapter.watchdog.file-edit-counter
adapter.watchdog.repair-cap
adapter.watchdog.out-of-scope-block
adapter.watchdog.version-bump-gate
adapter.watchdog.tracked-deletion-gate
adapter.watchdog.blocker-emitter
adapter.watchdog.monitoring-stream
adapter.watchdog.monitoring-stream-strict
adapter.post-install.executable-scripts-marked
adapter.post-install.beads-init-when-enabled
adapter.determinism.repeated-invocation-stable
adapter.mcp.install-implemented
adapter.mcp.remove-implemented
adapter.mcp.list-implemented
adapter.sync.pre-sync-implemented
adapter.sync.classifies-all-files
adapter.doctor.implemented
adapter.doctor.verifies-cli-installation
adapter.doctor.verifies-required-files
adapter.doctor.runs-conformance-subset
adapter.output-styles.emitted

B.2 Stack pack checks (~18)

stack-pack.manifest.declared-level
stack-pack.manifest.languages-non-empty
stack-pack.manifest.spec-version
stack-pack.detect.returns-result
stack-pack.detect.confidence-categorical
stack-pack.detect.no-shell-calls
stack-pack.detect.handles-missing-files
stack-pack.skills.at-least-one-skill
stack-pack.skills.source-marked
stack-pack.skills.has-review-markers
stack-pack.skills.no-secrets
stack-pack.skills.no-hard-coded-paths
stack-pack.determinism.repeated-invocation-stable
stack-pack.mcp.references-catalog-ids
stack-pack.mcp.no-sensitive-suggestions
stack-pack.validation.commands-well-formed
stack-pack.validation.ci-safety-marked
stack-pack.validation.respects-project-scripts
stack-pack.architecture.pattern-implemented
stack-pack.architecture.suggested-layout-non-empty
stack-pack.composability.composes-with-declared

See also