aidokitwiki
User Docs Wiki

How to run tests and byte-compare

Audience-Contributor Status-Shipped v0.5

Purpose #

Run unit, integration, conformance, and dogfood-byte-compare tests and interpret failures.

All packages, default tiers #

pnpm test

Turborepo orders tests after each package's build; cached package tests skip if inputs unchanged.

Single package #

pnpm --filter @aidokit/<pkg> test
pnpm --filter @aidokit/adapter-claude-code test
pnpm --filter aidokit test

Just a single test file #

pnpm --filter aidokit test test/integration/init-emit.test.ts

Conformance harness (per package) #

Each adapter and stack-pack package ships a conformance.test.ts. Run via that package's test script:

pnpm --filter @aidokit/adapter-claude-code test
# includes runAdapterConformance(adapter, 'strict')

To run only the conformance test:

pnpm --filter @aidokit/adapter-claude-code test test/conformance.test.ts

Dogfood byte-compare #

The strongest integration test. From a clean built tree:

cd /tmp && rm -rf dog && mkdir dog && cd dog && git init
node ~/aidokit/packages/cli/dist/bin/aidokit.js init \
  --adapter claude-code --stack node-ts --yes

diff -r ~/aidokit/.claude  ./.claude

Expected: empty output (clean). Any output is a diff that must be reconciled.

For automation (CI mirrors this):

diff -r ~/aidokit/.claude ./.claude > /tmp/dog-diff.txt
[ ! -s /tmp/dog-diff.txt ] && echo "DOGFOOD CLEAN" || (cat /tmp/dog-diff.txt; exit 1)

E2E #

Gated behind a separate script:

pnpm test:e2e
?
TODO

Confirm with maintainer — current state of the e2e/ lane and whether it's wired into CI.

Coverage #

Targets (CLAUDE.md §7.4):

Inspect with Vitest's coverage flag:

pnpm --filter @aidokit/core test --coverage

Interpreting common failures #

Failure Likely cause
adapter.emit-engine.paths-under-engine-dir fails An emit method returned a file outside the adapter's engine dir.
adapter.determinism.repeated-invocation-stable fails A non-deterministic field (timestamp, random) in some emission.
stack-pack.detect.no-shell-calls fails detect() reached for child_process instead of DetectContext.
Integration test asserts ≥18 skills, got fewer New stack pack didn't load, or @aidokit/base-skills regressed.
Dogfood compare: Only in <emitted>: foo.md Adapter emits a file the hand-built .claude/ lacks. Add to hand-built or remove from emission.
Dogfood compare: Files differ: settings.json Adapter and hand-built diverge in content; reconcile one side.
Dogfood compare: mode differs An executable script lost or gained the executable bit.
AD-STD-CAP-01 fails Adapter imports child_process or calls fetch(/http(s).request( but the corresponding manifest field is empty.
MCP_INSTALL_FAILED for graphify in init E2E Python prereq absent. Expected per ADR-0015; rest of scaffold should still complete.