CI Overview + Validator Catalog

This document maps every GitHub Actions workflow + every validator script in pm-skills + how they fit together. Use it when:

You’re a contributor wanting to understand what CI will check before you open a PR
You’re a maintainer auditing why a check fired (or didn’t fire)
You’re authoring a new validator and want to know where to wire it
You’re triaging a CI failure and want to know which workflow file owns which check

Three Layers of CI in pm-skills

pm-skills CI has three layers:

Local pre-commit / pre-push (developer machine; optional). Run via individual script invocation or the scripts/pre-tag-validate.{sh,ps1} orchestration bundle.
GitHub Actions on PR (automatic; every push to a non-default branch + every PR open/update). The 8 workflow files in .github/workflows/.
GitHub Actions on main + tag (automatic on push to main + tag push). Same workflow definitions; some have additional triggers like release: created for the doc-deploy + release-zips paths.

Most validators that fail in layer 2 would have caught the issue in layer 1 if run locally. Maintainers are encouraged to run pre-tag-validate before every release-prep commit.

flowchart TD
    PushPR[Push to non-main branch<br/>or PR open/update] --> Validation[validation.yml<br/>ubuntu + windows]
    PushPR --> Plugin[validate-plugin.yml]
    PushPR --> CodeQL[codeql.yml]

    PushMain[Push to main<br/>docs/ path filter] --> Pages[deploy-pages.yml]
    PushMain --> Drafts[create-issues-from-drafts.yml]

    TagPush[Tag push v*.*.*] --> Release[release.yml]
    TagPush --> Zips[release-zips.yml]

    Manual[workflow_dispatch] --> SyncAgents[sync-agents-md.yml]

    Validation --> Enforcing[enforcing validators:<br/>lint-skills-frontmatter<br/>validate-agents-md<br/>validate-commands<br/>family validators<br/>validate-docs-frontmatter --strict<br/>check-no-body-h1 --strict<br/>check-count-consistency<br/>check-skill-cross-references<br/>gen + astro build<br/>check-rendered-links + anchors<br/>check-route-parity<br/>+ more]
    Validation --> Advisory[Advisory validators:<br/>check-frontmatter-yaml]

    Enforcing -->|all pass| Green[PR green; merge unblocked]
    Enforcing -->|any fail| Red[PR red; fix + push again]
    Advisory -.->|informational| Green

    Pages --> Site[Astro 6.3.x build<br/>publish to GitHub Pages]
    Release --> GH[Publish GitHub Release<br/>auto notes + manual edit]
    Zips --> Artifacts[Attach zips to Release]

    Local[Local pre-commit:<br/>pre-tag-validate.sh / .ps1] -.->|runs the validator scripts| Enforcing

The diagram shows the trigger -> workflow -> validator -> outcome flow. The dotted arrow from pre-tag-validate to the enforcing validators indicates the local-machine equivalent: running the bundle locally exercises the same validator scripts the CI runner runs. The bundle does NOT run the Astro build (deploy-pages.yml), plugin-install validation (validate-plugin.yml), edit-link verification, or cross-doc reference checks, so a green local bundle is a strong - but not complete - predictor of a green PR check.

GitHub Actions Workflows

All 8 workflow files in .github/workflows/:

Workflow	Trigger	What It Does
`validation.yml`	Push to any branch + PR open/update	Runs the full enforcing validator suite on ubuntu-latest + windows-latest. This is the primary CI gate.
`validate-plugin.yml`	Push + PR + workflow_dispatch	Validates the plugin manifest at `.claude-plugin/plugin.json` + the marketplace manifest at `.claude-plugin/marketplace.json` + commands + skill structure.
`sync-agents-md.yml`	Workflow_dispatch only	Manual maintainer trigger to regenerate AGENTS.md from skill directory state. Hardened with two-layer defense (apply:true input gate + contents:read token gate).
`codeql.yml`	Push + PR + schedule	CodeQL static analysis for security + code quality.
`create-issues-from-drafts.yml`	Push to main (issue-draft path filter) + workflow_dispatch	Auto-creates GitHub issues from `docs/internal/issue-drafts/` when they merge to main.
`release.yml`	Tag push (`v..*` pattern) + workflow_dispatch	Publishes the GitHub Release with auto-generated notes (maintainer edits the body post-tag for full notes).
`release-zips.yml`	Tag push (`v..*` pattern)	Builds the release zip artifacts and attaches them to the GitHub Release.
`deploy-pages.yml`	Push to main (docs path filter) + workflow_dispatch	Builds the Astro 6 + Starlight 0.39.x doc site and deploys to GitHub Pages at https://product-on-purpose.github.io/pm-skills/.

Which Workflow Runs Which Validators

The validation.yml workflow contains the lion’s share of validator script invocations (~56 across both ubuntu + windows jobs). The validate-plugin.yml workflow contains ~12 plugin-specific script invocations. Together they cover the full enforcing surface.

codeql.yml, deploy-pages.yml, release.yml, and release-zips.yml do not invoke pm-skills validators - they do specialized jobs (security scanning, doc-site build, release publishing).

Validator Catalog

Validators live in scripts/. Most have three sibling files: {name}.sh (POSIX bash), {name}.ps1 (PowerShell), and {name}.md (documentation). This is the canonical .sh + .ps1 + .md convention.

Enforcing Validators (truly block CI on failure)

These run in validation.yml and fail the build if they exit non-zero.

Validator	What It Catches
`lint-skills-frontmatter`	Skill YAML frontmatter consistency (name, classification, version, license, metadata, phase when applicable)
`validate-agents-md`	AGENTS.md skill paths + sub-agent names in sync with `skills/` and `agents/` directories (extended in v2.16 to recognize sub-agent directory)
`validate-commands`	Each command file under `commands/` references a valid `skills/*/SKILL.md` path
`validate-meeting-skills-family`	Meeting Skills Family contract enforcement (5 members; v2.11.0)
`validate-foundation-sprint-skills-family --strict`	Foundation Sprint Family contract enforcement (7 members; v2.15.0)
`validate-design-sprint-skills-family --strict`	Design Sprint Family contract enforcement (7 members; v2.15.0)
`check-rendered-links` (post-build)	Every internal link in the BUILT site (`site/dist/`) resolves to a real route, and every `#fragment` resolves to a real element id (anchors enforced via `STRICT_ANCHORS=1` in CI; advisory otherwise). The build-aware successor to the retired filesystem link validators: `scripts/remark-resolve-links.mjs` emits Starlight slug links + GitHub source URLs at build, and this gate proves zero are broken
`check-route-parity` (post-build)	No published URL silently disappears. Compares the built route set (`site/dist/*/.html`) against the committed baseline `scripts/route-manifest.txt` and fails if any baseline route is missing (a removed/renamed URL is a “Site not found” for existing links/bookmarks). Added routes are allowed. After an intentional route change, add a redirect (keeps the route present) or regenerate the baseline with `node scripts/check-route-parity.mjs --update` and commit it with a reason
`validate-docs-frontmatter --strict`	Astro Starlight frontmatter shape on hand-authored docs in `site/src/content/docs/` (generated subtrees auto-skipped)
`check-no-body-h1 --strict`	No body H1 duplications (Starlight derives H1 from `title` frontmatter; body H1 would duplicate)
`check-count-consistency`	Skill/command/workflow counts in tracked .md, .mdx, and .json match filesystem state, including the `badge/skills-<N>` shields-badge form (.mdx + badge added v2.19.0)
`check-skill-cross-references`	Backtick skill-name references in `skills//SKILL.md` resolve to a real `skills//` directory; intentional forward-refs are allowlisted (v2.19.0)
`check-skill-page-sections`	Every `## section` in a `SKILL.md` renders on its generated docs page. Output-based: it imports `renderSkillPage` from `scripts/gen-site.mjs`, renders each page, and fails if any section is missing. Guards the silent-section-drop class that shipped ~27 hollow skill pages before v2.29.1
`check-sample-counts`	The on-disk library sample + sampled-skill counts match the headline numbers in `README_SAMPLES.md` and the site samples landing page (`check-count-consistency` excludes `library/`, so this closes that gap; v2.29.0). Also asserts the per-thread distribution and, since the orbit/legacy retirement, thread-classification agreement: every sample resolves to one of the three canonical threads. A missing or misspelled `thread:` is invisible to the distribution-versus-total reconciliation, because the outside bucket is one of its own addends
`gen-derived-surfaces --check`	Two gates in one step. (a) Every generated count surface (README catalog regions, the three manifest description headlines, the marketplace release pin, the site changelog mirror, the releases index) matches what the generator would emit, so a hand-edit inside a generated region fails CI. (b) C2, manifest-tail freshness: the AUTHORED tail of each manifest description still opens on the shipped version. That tail is prose the generator carries through verbatim, which is exactly why nothing read it and why all three tails still pitched v2.32.0 two days after v2.33.0 shipped. Only the LEADING version token is asserted: a tail may legitimately narrate an older release further along, and flagging every token in sight is the heuristic that got `check-version-references` retired at 1287 findings
(generated content: completeness-guarded)	Reference pages are emitted by `scripts/gen-site.mjs` into `site/src/content/docs/` and are gitignored + rebuilt each build, so there is no committed drift surface; content completeness is guarded by `check-skill-page-sections` (above)
`check-landing-page-counts --strict`	Landing-page total count claims (`site/src/content/docs/index.mdx`, `skills/index.md`, etc.) match filesystem state
`check-workflow-generator-coverage`	Every workflow source has both an individual page and an index-table row
`check-agents-md-command-sync`	AGENTS.md command table is in sync with `commands/` directory
`validate-script-docs`	Every `scripts/.sh` + `.ps1` pair has a companion `*.md` doc (promoted to enforcing v2.19.0, FU-8)

Advisory Validators (informational; do not block)

Validator	What It Surfaces
`check-mcp-impact`	Detects pm-skills changes that may require pm-skills-mcp companion updates (M-22 maintenance-mode posture; never blocking)
`check-frontmatter-yaml`	YAML parse validity in frontmatter (sibling to lint-skills-frontmatter; finer-grained on YAML errors)
`check-generated-freshness`	Generation-timestamp recency check
`check-workflow-coverage`	Older variant of workflow-generator-coverage; complementary checks
`check-em-dashes`	Em-dash + en-dash sweep (CLAUDE.md hard rule enforcement)

Post-Tag Delivery Check (deliberately NOT in CI or the pre-tag bundle)

Check What It Catches

check-delivery-pin.mjs C1. Whether the product-on-purpose marketplace users actually install from serves the tag just pushed. pm-skills ships through TWO repositories: that marketplace lives in agent-plugins and pins members by commit SHA independently of anything here. On 2026-09-01 v2.33.0 was tagged, released and fully green while reaching nobody for about eight hours, because the registry still pinned the v2.32.0 commit. A stale SHA pin resolves to real, working, older code, so nothing errors and no other check has an opinion. Compares BOTH the version label and source.sha: a registry that moved the label but not the pin serves old code under a new number

Check	What It Catches
`check-delivery-pin.mjs`	C1. Whether the `product-on-purpose` marketplace users actually install from serves the tag just pushed. pm-skills ships through TWO repositories: that marketplace lives in `agent-plugins` and pins members by commit SHA independently of anything here. On 2026-09-01 v2.33.0 was tagged, released and fully green while reaching nobody for about eight hours, because the registry still pinned the v2.32.0 commit. A stale SHA pin resolves to real, working, older code, so nothing errors and no other check has an opinion. Compares BOTH the version label and `source.sha`: a registry that moved the label but not the pin serves old code under a new number

Why it is not in CI or the bundle. The pre-tag bundle runs before the tag exists, so the question is unanswerable rather than merely unasked. validation.yml triggers on push to main and on pull requests, where there is no new tag to ask about, and immediately after a real tag it would fail for hours by design, because the re-pin is a separate cross-repo pull request in another repository.

Why it does not block a single sitting. The real v2.33.0 tag-to-repin gap was about eight hours. A gate that must go green inside one G4 walkthrough would be routed around on first use, and a gate that is routinely bypassed is worse than none. It is a cheap, idempotent question you re-ask until the answer is yes. Exit 0 delivered, exit 1 tagged-but-not-delivered, exit 2 unverifiable. It never prints the success line on exit 2: an unanswered delivery question blocks “Release complete” exactly as a failed one does, because silence being read as success is the original defect.

Orchestration Bundles

Bundle	What It Runs
`pre-tag-validate.{sh,ps1}`	The full enforcing validator suite + a configurable advisory bundle. Recommended before any release-prep commit.
`embed-skills.js` (in pm-skills-mcp)	Companion script in the pm-skills-mcp repo; reads pm-skills `skills/` directory and embeds skill metadata into the MCP server. NOT a validator but mentioned here because it consumes pm-skills validator output indirectly.

How to Trigger Each Workflow Locally

Workflow	Local Equivalent
`validation.yml`	`bash scripts/pre-tag-validate.sh` (or `.ps1` on Windows)
`validate-plugin.yml`	`bash scripts/validate-plugin.sh` if present, else individually invoke `validate-commands.sh` + frontmatter checks
`codeql.yml`	Run CodeQL CLI locally per CodeQL docs
`deploy-pages.yml`	`npm run build` (Astro production build)
Others	Manually inspect the workflow_dispatch trigger inputs; some have no local equivalent

Windows local-run notes:

Node on PATH. In Git Bash on Windows, Node is not always on PATH. If cd site && npm run build or the node-based checks (gen-site.mjs, verify-edit-links.mjs, check-rendered-links.mjs) fail with node: command not found, prepend Node for the session: export PATH="/c/nvm4w/nodejs:$PATH" (Node 24). The bash validator scripts do not need Node; only the build + generator + link-check steps do.
Line endings. .gitattributes (added v2.19.0) pins *.sh to LF so Git Bash runs them without CRLF false-reds, regardless of your core.autocrlf setting. *.ps1 is checked out as CRLF; *.mjs/*.js/*.json as LF.

How to Add a New Validator

When adding a new validator script:

Write it as single-source Node: scripts/{name}.mjs plus a sibling scripts/{name}.test.mjs. The dual-shell inventory has been FROZEN since v2.30.0 (see the DUAL-SHELL FREEZE note in scripts/validation-manifest.yaml): do NOT add a new .sh + .ps1 pair. The existing pairs carry roughly 3,700 duplicated lines, and the parity referee proves only that the two inventories match, never that the two shells reach the same verdict. Export the logic as pure functions with their I/O injected, following scripts/check-sample-counts.mjs, so the test can prove every branch without touching the real repo.
Ship a fixture that proves the check FIRES, not only that it passes. A test that cannot fail converts an open question into a checked box. This is a standing rule of the documentation currency program.
Wire it into .github/workflows/validation.yml. That workflow is ONE validate job with an OS matrix, so a Node validator needs a single step and runs on both legs automatically. Only the legacy shell pairs carry if: matrix.os == ... guards.
Wire it into scripts/pre-tag-validate.{sh,ps1} ONLY if it can be answered before a tag exists. A check that is only meaningful after the tag belongs at G4 instead, like check-delivery-pin.mjs. Note pre-tag-validate.sh runs under set -u, where expanding an empty array is fatal on bash before 4.4, so a tier with no members must be removed outright rather than left empty.
A .md sidecar is for shell pairs only: validate-script-docs iterates .sh and .ps1 bases, and scripts/README_SCRIPTS.md catalogs shell utilities. A Node validator self-documents in its header comment.
Update this document’s validator catalog (the relevant section above) with the new entry.
Before adding one at all, weigh it against the counter-metric in the documentation currency program: the enforcing suite is not meant to grow by more than four checks. Extending an existing validator, as C2 did inside gen-derived-surfaces, costs nothing against that budget.
Add a test case to docs/internal/release-plans/v{next-version}/testing-summary_v{next-version}.md if it would be useful to include in the next release’s testing investment record.

CI Failure Triage Pointers

Failure Symptom	First Place to Look
validation.yml red, ubuntu-latest only	Linux-specific shell-script issue (e.g., bash version, line ending)
validation.yml red, windows-latest only	PowerShell-specific issue (e.g., encoding, CRLF parsing, `-p` flag)
validate-plugin.yml red	Plugin manifest schema; check `.claude-plugin/plugin.json` validity + commands paths
`check-rendered-links` red	A link in the built site does not resolve; check the source link and how `scripts/remark-resolve-links.mjs` maps it (in-site slug vs GitHub source URL). If it names a broken `#anchor`, the heading id moved or the fragment is stale
`check-route-parity` red	A previously-published URL is gone from the build; restore it, add an Astro redirect (keeps the route present), or - if the removal is intentional - run `node scripts/check-route-parity.mjs --update` and commit the new `scripts/route-manifest.txt` with a reason
`validate-docs-frontmatter --strict` red	Astro Starlight schema requires `title:` + ASCII colons in description; check the offending doc
`check-no-body-h1 --strict` red	A body `# Header` line exists in a hand-authored page; remove or convert to `##`
`npm run build` red (generator)	`scripts/gen-site.mjs` failed or emitted invalid frontmatter; run `cd site && npm run build` locally and read the Astro error
`check-count-consistency` red	A skill/command/workflow count claim in tracked .md is out of date; search for the stale number and update

Reference Links

All workflow files: .github/workflows/
All validator scripts: scripts/
Pre-tag validator bundle: scripts/pre-tag-validate.sh + .ps1
v2.16.0 test summary (concrete example of all validators in action): docs/internal/release-plans/v2.16.0/testing-summary_v2.16.0.md
Authoring sub-agents (relevant for sub-agent-specific validators in v2.17): docs/contributing/authoring-sub-agents.md

Versioning

Doc version	Date	Change
1.0.0	2026-05-17	Initial publication; consolidates the CI workflow + validator catalog that was previously scattered across `validation.yml` comments, individual `scripts/*.md` files, and release-plan testing summaries.