Gold (Advanced) conformance checks
Reference: Gold (Advanced) conformance checks
Section titled “Reference: Gold (Advanced) conformance checks”The Gold tier adds requirements G1-G10 (Standard sec 2.6) on top of Bronze + Silver. Each fires findings tagged reqId: "G<n>"; tier-report buckets them into the advanced tier and lists unmet ones in blocked.advanced. Because they are advanced-tier, they do NOT fail the gate for a plugin that declares universal or convergent - they appear as the burndown to Gold until the plugin declares advanced and addresses them. The full Gold set G1-G10 is implemented; this reference documents every check.
| reqId | Module | What it checks | Standard | Conditional? | Example fix |
|---|---|---|---|---|---|
| G1 | scripts/checks/hook-documentation.mjs | Every hook in hooks/hooks.json documents its structure: a type per action, and a matcher for tool-matched events (PreToolUse/PostToolUse) | sec 2.6 (G1), sec 3.5 | yes (hooks exist) | Add the missing type (command/http/mcp_tool/prompt/agent) to each hook action and a matcher to each PreToolUse/PostToolUse entry. |
| G2 | scripts/checks/self-hosting.mjs | A CI workflow under .github/workflows/ runs the conformance gate (node scripts/check.mjs) - the plugin passes its own validators | sec 2.6 (G2), sec 4 | no | Add a CI workflow that runs node scripts/check.mjs (and npm test). |
| G3 | scripts/checks/library-regression.mjs | Each chain edge (in agents/_chain-permitted.yaml) and each hook event (in hooks/hooks.json) carries at least one eval/regression case under evals/; an eval covering an edge the contract no longer permits is a stale case (the regression signal) | sec 2.6 (G3), sec 8.3 | yes (a chain contract or hooks exist) | Add evals/<name>.eval.json with "covers": { "chain": ["<caller>", "<callee>"] } for each uncovered chain (or { "hook": "<event>" } for a hook). Remove or update a stale eval whose covered edge is gone. |
| G4 | scripts/checks/index-drift.mjs | INDEX.md matches what gen-index produces from library.json + component frontmatter (the native manifests + manifest.generated.json are already drift-checked by U8) - a hand-edited generated INDEX is an error | sec 2.6 (G4), sec 10.3 | no | Regenerate: node scripts/generators/gen-index.mjs . --write. Edit the source (frontmatter / library.json / gen-index), not INDEX.md. |
| G5 | scripts/checks/release-notes.mjs | A curated, user-facing RELEASE-NOTES.md exists at the root, distinct from CHANGELOG.md | sec 2.6 (G5), sec 10.6 | no | Add RELEASE-NOTES.md (highlights-first, user-facing); askit-release notes mode curates it. |
| G6 | scripts/checks/deprecation.mjs | Every library.json component status is valid, and a deprecated component declares deprecated-by (its replacement) and remove-in (its removal version) | sec 2.6 (G6), sec 3.7, 7.5 | yes (a deprecated/invalid status exists) | On the deprecated component’s library.json entry add "deprecated-by": "<replacement>" and "remove-in": "<version>", or correct an invalid status (active/deprecated/experimental). Manage it with askit-deprecate. |
| G7 | scripts/checks/docs-frontmatter.mjs | Every published docs/** page (excluding docs/internal/) carries the frontmatter taxonomy: title, description (no colon-space), audience in {non-engineer, engineer, both}, level in {beginner, intermediate, advanced}, optional tags/doc-role | sec 2.6 (G7), sec 8.4 | yes (a published docs tree exists) | Add the missing or out-of-vocabulary frontmatter field to the flagged page; author docs with askit-build-docs. |
| G8 | scripts/checks/folder-readme.mjs | Every meaningful folder has a README.md with a frontmatter title and an ## Inventory whose listed immediate children set-equal the folder’s actual children (reports under-listed + phantom) | sec 2.6 (G8), ADR 0024 D1.1 | yes (an allowlisted folder exists) | Add or refresh the folder README.md so its inventory lists every immediate child; use askit-build-docs (folder-readme mode). |
| G9 | scripts/checks/source-doc.mjs | Every hand-authored *.mjs/*.js/*.py under the in-scope roots (scripts/, site/scripts/, hooks/) carries a four-field header docblock (what-it-is / what-it-does / why / used-by) in its first lines | sec 2.6 (G9), ADR 0024 D1.2 | yes (in-scope source exists) | Add the four labeled comment lines (or the @what/@does/@why/@usedby tags) to the file header. |
| G10 | scripts/checks/docs-presence.mjs | The four Diataxis quadrants are non-empty, every ADR (docs/internal/decisions/NNNN-*.md) carries a ## TL;DR, and the doc-role: architecture-overview page links the architecture-detailed page | sec 2.6 (G10), sec 10.4 | yes (a docs/ tree exists) | Populate the empty quadrant / add the missing ADR ## TL;DR / add the overview-to-detailed link. |
G3 is the baseline (presence + execution + the deterministic regression signal). The Standard explicitly defers the multi-tier eval engine (Static / LLM-Judge / Monte-Carlo) to roadmap; that judging layer ships beside the gate as opt-in evidence, never inside the deterministic CI gate (Design Principle 3, ADR 0023).
Eval-set format (evals/*.eval.json)
Section titled “Eval-set format (evals/*.eval.json)”Each file declares what it covers and (optionally) its cases. The deterministic check validates the covers declaration and references; the behavioral layer (opt-in) executes the cases.
{ "covers": { "chain": ["caller-component", "callee-component"] }, "description": "One line: the chained behavior this set exercises.", "cases": [ { "given": "the input or situation", "expect": "the behavior the chain should produce" } ]}covers is one of: { "chain": ["caller", "callee"] } (a permitted chain edge), { "hook": "<event>" } (a registered hook event), or { "skill": "<name>" } (a triggering eval set; a Universal SHOULD per sec 8.3, not gated by G3). The templates/eval-set.json skeleton scaffolds one.
See the add-eval-coverage how-to.