Test Strategy
beta · Family standing-standards · Phase undefined · Sizes lean · ~5,650 tokens
The standing document that sets the default testing approach for every project inside one organization or program: which levels, types, tools and environments apply by default, and the rule by which each project’s own test plan either follows that default or states a reasoned deviation. It is not a test plan, a master test plan, or a Definition of Done.
The short card. Why the document is shaped this way, and the argument behind every rule here, is in
test-strategy_companion.md. A fully worked instance is
test-strategy_example.md.
When to use
Section titled “When to use”- More than one project inside one organization or program needs the same testing defaults, and you want those defaults written down once rather than argued out fresh inside every project’s own test plan.
- You need a single place that states which levels or types are mandatory because of a law, a regulation, a contract, or a past incident, rather than a preference, so that obligation does not quietly get forgotten from one project to the next.
- A project’s own test plan already exists, or is about to be written, and needs a standing document to
name in its
test_strategy_reffield, inheriting this strategy’s defaults rather than inventing its own. - You want a single, named answer to who owns testing decisions at the organization or program level, kept distinct from who actually executes testing on any one project inside that scope.
- You want the standing document itself to name the specific events that would make it wrong, a recorded deviation, a changed test policy, a skipped mandated type, a new tool adopted without the document changing, each paired with a named owner, rather than a calendar date nobody reads before renewing it.
- Nobody has written down who approves a change to this organization’s or program’s testing defaults, and you want that named rather than left to be assumed by whoever happens to be in the room.
When NOT to use
Section titled “When NOT to use”- You are planning testing for one project, release, or effort. That is
test-plan, scoped to a single effort. It already carries the seam: its owntest_strategy_reffield names this strategy, and its Risk-Ranked Approach section records any deviation with the reason. This document must not restate either oftest-plan’s own sections. - You need to coordinate multiple test levels or test types for one project. That is a master test plan, which ISTQB defines directly as “a test plan that is used to coordinate multiple test levels or test types.” A master test plan sits beneath this strategy, coordinating one project’s own levels, not above it.
- You want a standard every increment of work is judged against, in any discipline, on a cadence. That is a Definition of Done. Both are standing, both are agreed once and consulted repeatedly rather than authored per increment, but a Definition of Done governs whether an increment is finished and this document governs the testing approach a project either follows or states a reasoned deviation from. A team may hold both at once.
- You need the entry-side standard for when a backlog item is ready to start. That is a Definition of Ready. It governs when work may enter a sprint; this document governs how testing is approached once work is already underway. Neither gates the other.
- You are checking one service’s standing fitness to be handed off and run day to day. That is a production readiness review. Its own criteria may ask whether a project’s testing conformed to this strategy, where one exists, without this bundle restating any of that review’s own criteria.
- You are specifying one effort’s own test cases, reporting its own test run, or filing a bug. Those
are
qa-docsfamily members (test-case,test-summary-report,bug-report), each authored per increment and finished at a stage. This document is none of those three things: it is not authored per increment and is not finished at any one stage. - You mean “test strategy” as a per-effort thinking tool rather than a standing, cross-project document. A heuristic model meant to be reworked for one effort, or a set of testing quadrants rebuilt every iteration, is a real and useful practice, but it is not this bundle’s subject: this bundle covers only the standing sense that spans more than one project.
Pick a variant
Section titled “Pick a variant”This bundle ships one size, lean, and there is no second, heavier variant to choose between. That is a
deliberate call, not an unfinished one: no source read for this bundle describes a short form and a long
form of a test strategy, and the sources that address size at all argue against a second, heavier weight
rather than for one. BrowserStack names “Overdocumenting Everything” as one of its own six named failure
modes for this kind of document: “Excessively detailed documents increase maintenance effort and are
rarely followed during actual testing.” Mirjana Kolarov’s own conference abstract names the image most
readers already hold, “A big document of 40+ pages with ton of theory, no one reads and is not up to
date,” and answers it by arguing that the document must be tailored and short, not by arguing for a second,
bigger shape. A full variant here would model the exact failure this bundle warns against.
If your situation feels like it needs more sections than the seven this bundle ships, take that feeling
seriously, but read it as a sign that this organization or program may need more than one strategy, the
way one vendor’s own glossary cautions an organization can hold more than one for markedly different
project contexts, rather than as a reason to look for a heavier template. See
test-strategy_companion.md section 4 for the full
reasoning.
Quality rubric (self-grade)
Section titled “Quality rubric (self-grade)”Score each section 0, 1 or 2. Below 10 out of 14 and a project team reading this strategy cannot tell what it is actually bound to follow, and falls back to inventing its own testing defaults project by project, which is the exact failure a standing, organization-or-program strategy exists to prevent.
| # | Criterion | 0 | 1 | 2 |
|---|---|---|---|---|
| 1 | Scope is bounded | No organization or program is named, or the scope reads as though it covers every project with no stated boundary | An organization or program is named and the projects it binds are stated, but nothing is named as explicitly out of scope | The organization or program is named, the projects it binds are stated, and a reader can point at the named excluded program, project, or release train and say why it is excluded |
| 2 | Mandatory rows are sourced | The table names levels or types with no default-or-mandatory distinction, or a mandatory row names no reason | Some rows distinguish default from mandatory, but a legal, regulatory, or contractual obligation that applies to this organization is missing, or a row is left “to be defined” | Every row states whether it is a default or mandatory, and every mandatory row names the law, regulation, contract, or past incident behind it, so a project reading it knows this is not a preference |
| 3 | Tooling and data rule | No default tools or environments are named, or no rule is given for real or production-derived data in test | Default tools or environments are named and a rule against production data exists, but the rule states no documented path for when synthetic data cannot do the job | Default tools and environments are named, the rule for real or production-derived data is stated, and a documented exception path, who decides and what gets written down, is stated for when synthetic data cannot suffice |
| 4 | Governance and approval | No relationship to a test policy is stated, and no one is named who approves a change to this strategy itself | One of the two is stated, but the approver named is a vague noun rather than a role, function, or committee a reader could actually go contact | The relationship to any test policy, including its stated absence, is given, and the approver of a change to this strategy is a specific role, function, or committee someone could actually reach |
| 5 | Inheritance and deviation | No statement of how a project adopts this strategy, or of what it must do to deviate from it | Adoption is named, but a deviation is described without saying who may accept it | A project’s adoption mechanism is named, a deviation’s required reason is stated, and the role that may accept a deviation is named, without inventing a mandated approver title no program-level source actually supports |
| 6 | Ownership is distinct | No role owns this document at the organization-or-program level, or the table only names who executes testing on one project | An owner is named, but the table does not distinguish that role from who executes testing on any one project in scope | A document owner at the organization-or-program level is named and distinguished from per-project testing roles, and a reader can point at a second named audience, someone who reads this document but neither owns nor executes against it, and say what they use it for |
| 7 | Review Trigger names conditions | Only a calendar date is named, or the table is blank | At least one condition is named, but a row carries no owner, or a periodic clause stands in as the entire trigger rather than a floor under a condition | Every row names a concrete condition someone could notice without being told to look, a named owner, and what that owner does about it, with any periodic clause present only as a floor layered under a condition |
The test behind every cell above: could someone satisfy it without improving the document? A row that counted rows, obligations, or conditions would reward padding a table. Every cell instead asks whether a specific, checkable fact is on the page, the kind a second reader, not the author, could point at.
Named anti-patterns (the usual wrecks)
Section titled “Named anti-patterns (the usual wrecks)”- Treating the strategy and the plan as the same document. BrowserStack names this directly: “Using
test plan and test strategy interchangeably creates confusion around testing scope, execution, and
ownership.” This document and
test-planare deliberately separate; collapsing them removes the exact distinction the inheritance mechanism depends on. - One document straining to cover every kind of project the organization runs. imbus’s own vendor glossary caution applies directly: an organization can hold more than one organizational test strategy “to cover markedly different project contexts.” A single sprawling document trying to fit every project type is a sign it should split, not a sign it is thorough.
- Overdocumenting into shelfware. Mirjana Kolarov’s own conference abstract names the image most readers already hold: “A big document of 40+ pages with ton of theory, no one reads and is not up to date.” BrowserStack names the same failure on its own list, “Overdocumenting Everything”: “Excessively detailed documents increase maintenance effort and are rarely followed during actual testing.”
- Copying a strategy unchanged between projects or organizations. TestRail warns against a strategy “copied unchanged between projects.” A strategy that was never tailored to this organization is not doing this document’s job, whatever it happens to be titled.
- Leaving a binding legal or regulatory obligation blank. The UK Office of the Public Guardian’s own published baseline leaves a shared field unanswered, “Accessibility To be defined,” beside the GDS Way’s own statement that the same obligation is a legal requirement. One instance is not a measured rate of this failure, but it is a real, preventable gap, and this bundle’s Test Levels and Types table exists partly to catch it before it ships blank.
- Inventing a named approver no source actually supports. The one source naming a required, titled approver for a deviation is a directive governing every test plan inside one agency, not a standing test strategy. Naming a mandated approval title for a deviation here manufactures a step the published program-level strategies never describe.
- A review trigger that is only a calendar date. TestRail states plainly, “There is no universal review schedule.” Where an annual or semi-annual clause is genuinely useful, it belongs as a floor layered under a named condition, never as the whole mechanism; a trigger that only says “reviewed annually” fires whether or not anything is actually wrong, and names no owner’s action.
- Missing or flattened ownership. BrowserStack names “Missing Clear Ownership” on its own list: “Without defined ownership, testing responsibilities and reporting processes become inconsistent across teams.” The sources this bundle read disagree on whether the owner is a QA function, a committee, or a project manager; flattening that live disagreement into an assumed single answer, rather than picking the shape that matches how this organization already makes its other standing decisions, is its own failure.
Pairing with your process
Section titled “Pairing with your process”This bundle ships in the standing-standards family alongside definition-of-done, definition-of-ready,
runbook, launch-coordination-checklist, production-readiness-review and business-continuity-plan,
as a foundation: a standard every project’s own test plan is judged against, or states a reasoned
deviation from, rather than an instrument executed under time pressure. test-plan’s own
test_strategy_ref field is the seam: a project names this strategy there and inherits its defaults,
recording any deviation inside its own Risk-Ranked Approach section with the reason. Update this file
when the organization’s or program’s own testing defaults actually change, not every time one project’s
plan records a single deviation; a recorded deviation is this strategy’s own signal to review, under the
Review Trigger above, not by itself a reason to rewrite the strategy on the spot.
The artifacts
Section titled “The artifacts”test-strategy_template-lean.md · ~5,650 tokens
---title: "{{organization_or_program_name}} Test Strategy"doc_type: test-strategysize: leanorganization_or_program_name: "{{organization_or_program_name}}"owner: "{{owner}}"status: draftdoc_version: "{{doc_version}}"created: "{{date}}"updated: "{{date}}"related_links: []source_template: test-strategysource_template_version: 0.1.0---
<!--LEAN TEST STRATEGY. This bundle ships one size, deliberately, and the evidence runs against a second,heavier weight rather than for one. Mirjana Kolarov names the image most readers already hold: "A bigdocument of 40+ pages with ton of theory, no one reads and is not up to date." BrowserStack names thesame failure as one of its own six anti-patterns, "Overdocumenting Everything". No source read for thisbundle describes a short form and a long form of this document; every structure found is onecomprehensive shape, not two weights of the same shape. A second, heavier variant would model the exactfailure this bundle warns against, so the document stays small on purpose. Seetest-strategy_companion.md section 4 (Variants and sizing).
WHAT THIS IS NOTIt is NOT a test plan. A test plan is scoped to one project and written per release; this document isstanding and spans more than one project in one organization or program. The seam is thetest_strategy_ref field already shipped in test-plan's own frontmatter: a project names this strategythere, inherits its defaults, and states any deviation, with its reason, inside its own Risk-RankedApproach section. This document must not restate either of test-plan's own sections.It is NOT a master test plan. ISTQB defines that type directly: "A test plan that is used to coordinatemultiple test levels or test types." A master test plan sits beneath this strategy, coordinating oneproject's own levels, not above it.It is NOT a Definition of Done. Both are standing, both are agreed once and consulted repeatedly ratherthan authored per increment. A Definition of Done is a standard every increment is judged against, in anydiscipline; this document is the testing-specific standard a project's own verification approach eitherfollows or states a reasoned deviation from. A team may hold both at once.See test-strategy_companion.md section 8 (Relationships to other artifacts).
HOW TO FILL THIS IN1. Read the comment under each heading: WHAT it wants, WHY it matters (with a pointer into test-strategy_companion.md for the deep reasoning), guiding questions to ASK, a GOOD and a WEAK example, and the TRAP to avoid. The three tables also carry PRIORITY and ROW HINT.2. Replace each {{placeholder}} with your content.3. If a section, or one row inside a table, does not apply, write "N/A" and one line of why, rather than leaving it blank or deleting the section. An unexplained blank and a skipped item look identical to a later reader.4. This is a standing document, revisited on the Review Trigger below, not written once and forgotten. Before you first share it: self-grade against test-strategy_guide.md, then DELETE every HTML comment. They are guidance, not content.-->
# {{organization_or_program_name}} Test Strategy
## Purpose and Scope
<!-- WHAT What counts as "this organization or program" here, which projects it binds, and what it explicitly leaves out, either to a separate strategy or to each project's own test plan. WHY Every published program strategy this bundle's research read draws a named boundary rather than leaving scope implicit. DCC's own joint test strategy names an explicit exclusion among several read in raw text: "Certification of Comms Hubs." A good strategy excludes by naming what it leaves out the way DCC does, rather than leaving a reader to guess. imbus's own caution against writing one strategy for every kind of project this organization runs: "An organisation could have more than one Organizational Test Strategy to cover markedly different project contexts." Scope this document narrowly rather than writing one strategy to cover every project this organization or program runs. Deep dive: test-strategy_companion.md section 3 (Anatomy > Purpose and Scope). ASK Which organization or program does this strategy bind? Which projects inside it are in scope? What is explicitly out of scope, the way DCC names an exclusion? Does a different kind of project inside this same organization need its own, separate strategy instead of being forced into this one? GOOD "This strategy binds every project run by the Billing Modernization program at Granite Peak Utilities. In scope: all projects under that program's own release calendar. Out of scope: the Field Operations program, which keeps its own separate test strategy, and third-party meter certification testing, which the vendor's own contract governs." WEAK "This is how we test." (names no organization or program, no bound projects, and nothing explicitly excluded) TRAP Writing one strategy to cover every kind of project this organization runs. imbus's own caution is that an organization can hold more than one Organizational Test Strategy for markedly different project contexts; a single document straining to fit all of them is a sign it should split, not a sign it is thorough. -->
{{purpose_and_scope_statement}}
**Explicitly out of scope:** {{explicit_exclusions}}
## Test Levels and Types
<!-- WHAT The levels and types every project in this organization or program inherits by default, and which of them are mandatory because of a law, a regulation, a contract, or a past incident rather than a preference. WHY No standards-tier source read for this bundle names a single required list of levels; a published vendor template asks only that an author name the levels and types a strategy covers, without prescribing one. Where an external obligation makes a type mandatory, naming the obligation belongs here. A UK government engineering manual states the duty plainly for public-sector services: "Government services are legally required to be accessible. This means services must comply with the international WCAG 2.2 AA accessibility standard." A US federal agency ties several types directly to law: "All system, program, or test plans must include verification strategies addressing system integration, acceptance, regression, privacy, and security as required by Cybersecurity and Section 508 of the 1973 Rehabilitation Act as currently amended." A standing regression rule is a third candidate evidenced directly, from a UK public body's own published baseline: "Regression should be built into automation at the unit and integration level" and "An automated regression test should be added whenever you find a bug." This section must not restate test-plan's own Test Levels and Types section or its Risk-Ranked Approach rows; it states the organization-or-program default, never any one release's own levels. Deep dive: test-strategy_companion.md section 3 (Anatomy > Test Levels and Types). ASK Which levels and types does every project in this organization or program run by default? Is any of them mandatory because of a law, a regulation, a contract, or a past incident, rather than a preference? Where does this strategy currently carry a blank where one of those obligations belongs? PRIORITY List any level or type mandated by a law, a regulation, a contract, or a past incident first; a team can get a preference wrong and adjust it later, but a mandated type left out is a different kind of failure. ROW HINT A good row names a level or type, states whether it is this program's own default or a mandatory requirement, and, if mandatory, names the obligation behind it. A weak row names neither a level nor an obligation. GOOD | Accessibility (WCAG 2.2 AA) | Mandatory | Legal obligation for UK public-sector services: "Government services are legally required to be accessible. This means services must comply with the international WCAG 2.2 AA accessibility standard." | WEAK | Testing | Default | | (names no level, no type, and no obligation) TRAP Leaving a binding obligation blank rather than researching it once and writing it down. A UK public body's own published baseline, otherwise filled in, states "Accessibility To be defined" beside a sibling obligation another source calls a legal requirement. A single instance is not a measured rate of this failure, but it is a real, preventable one. -->
| Level or Type | Default or Mandatory | Reason or Obligation ||---|---|---|| {{level_or_type}} | {{level_or_type_status}} | {{level_or_type_reason}} |
## Standard Tooling and Environments
<!-- WHAT The tools and environments every project is expected to use by default, and the rule for using real or production-derived data in any of them. WHY A published vendor template names establishing test environment and tooling requirements as its own step, without prescribing what they must be. A data protection regulator states the general principle a rule here should rest on: in testing, "sampling of real personal data should be avoided," with "artificially created test data" preferred, while conceding that where that cannot suffice, "a comprehensive decision must be taken and documented". A US federal agency's own live-data rule shows one working version of that documented exception: "If live data is used, prepare live data waiver, and ensure all testers have complete live data training." Deep dive: test-strategy_companion.md section 3 (Anatomy > Standard Tooling and Environments). ASK What tools and environments should a project reach for first, by default? What is the rule for using real or production-derived data in any of them? Is there a documented-exception path for when synthetic data cannot suffice, or does the rule simply forbid it? GOOD "Default environment: a staging tier seeded with synthetic data generated from the production schema, never production data itself. Where synthetic data cannot exercise a defect under investigation, a written, dated waiver naming the specific tables and the reviewer who approved it is required before any real data is loaded." WEAK "Use whatever environment you have." (names no default tools or environments, and says nothing about production-derived data) TRAP Writing a rule that only forbids production data, with no documented-exception path. A project that hits a case synthetic data cannot cover will then either break the rule quietly or stall for lack of an escape valve; a regulator's own guidance names the exception path as part of the rule, not as a separate concession. -->
{{standard_tooling_and_environments}}
**Rule for real or production-derived data in test:** {{production_data_rule}}
## Governance and Policy Alignment
<!-- WHAT How this strategy relates to any higher test policy, where one exists, and who approves a change to the strategy itself. WHY ISTQB's current glossary defines the policy that sits above this kind of strategy: "High-level documentation describing the principles, approach and major objectives of the organization regarding testing." A secondary account of the standard this strategy's own structure draws on states the alignment directly for this kind of document: "It is aligned with the Organizational Test Policy". Who approves a change to the strategy itself is answered differently across the sources this bundle's research read, and this section should show that disagreement rather than pick a winner: one vendor ties it to "QA managers, test leaders, and strategic teams"; another ties it to "a QA board or a cross-functional committee including IT, business and architecture representatives." A published program strategy instead routes approval to a named design assurance board. Deep dive: test-strategy_companion.md section 3 (Anatomy > Governance and Policy Alignment). ASK Does a test policy exist above this strategy? If so, name it and state how this strategy aligns with it. If not, say so plainly rather than leaving the question unanswered. Who approves a change to this strategy itself: a named role, a function, or a committee? GOOD "No separate test policy exists yet; this strategy is Granite Peak Utilities' first written statement of testing principles and will be revised to align the day a policy is adopted. Changes to this strategy are approved by the VP of Engineering, after review by the Billing Modernization program's own architecture group." WEAK "Approved by QA." (names no policy relationship, and no actual approving person, function, or body a reader could go find) TRAP Assuming one shape, a single manager, a function, or a committee, because one source's own account happens to describe it. The sources disagree on granularity; pick the shape that matches how this organization actually makes its other standing decisions. -->
{{test_policy_alignment_statement}}
**Who approves a change to this strategy:** {{strategy_approver}}
## How Project Test Plans Inherit or Deviate
<!-- WHAT The seam with test-plan's own test_strategy_ref field: what a project must do to adopt this strategy's defaults, and what it must do, and who may accept it, when it deviates. WHY Every program-level source this bundle's research read describes exactly this relationship. One published program strategy states, "This testing strategy will be followed by a more detailed programme level Testing Management Plan". It names what that subordinate plan carries too: "the detailed testing approach for each test phase, entry and exit criteria, detailed roles and responsibilities, reporting arrangements". A procurement schedule requires a project's own plan to state "the reasons and rationale for such differences" wherever it differs from this kind of strategy. A UK public body's own baseline calls a local variance "a conscious choice" that should "be able to be explained and preferably documented e.g. in an ADR." No source this research read states a required, named approver for a deviation from a standing strategy itself, so this section asks who may accept a deviation rather than naming a mandated title. Deep dive: test-strategy_companion.md section 3 (Anatomy > How Project Test Plans Inherit or Deviate). ASK What must a project do to adopt this strategy? (Name it in test-plan's own test_strategy_ref field.) What must it do to deviate from one of this strategy's defaults? Who may accept that deviation, by role rather than by name? GOOD "A project adopts this strategy by naming it in its own test plan's test_strategy_ref field. Any deviation from this strategy's levels, types, or tooling is recorded with its reason in that plan's own Risk-Ranked Approach section. The program's QA Lead may accept a deviation after reviewing the stated reason; this strategy does not require sign-off above that role." WEAK "Follow the strategy." (names no adoption mechanism, no deviation path, and no one who may accept a deviation) TRAP Naming a required, named approver title for a deviation on the strength of a single directive-level source. The evidence this research found supports asking who may accept a deviation, not mandating a title; adding one anyway invents an approval step no program-level source this bundle read actually names. -->
{{inherit_or_deviate_statement}}
**Who may accept a deviation from this strategy:** {{who_may_accept_a_deviation}}
## Roles and Responsibilities Across Projects
<!-- WHAT Who owns this document at the organization-or-program level, distinct from who executes testing on any one project inside it. WHY This recurs in every published program strategy this bundle's research read, independent of how each document is otherwise structured. Who owns the document is answered differently across the sources, and this section should show the disagreement rather than flatten it: one vendor ties ownership to "QA managers, test leaders, and strategic teams", while another source instead states, "Project Manager prepares Test Strategy." That directly contradicts the first. A job posting for a program-scoped role ties the artifact to that role instead, naming someone who will "Manage program testing activities that are correctly prioritised to deliver outcomes that align to the Departments priorities." The audience is wider than the owners: a practitioner source names "Project Sponsors, Business SMEs" among those this kind of document is written for, alongside several delivery teams. Deep dive: test-strategy_companion.md section 3 (Anatomy > Roles and Responsibilities Across Projects). ASK Who owns this document, separately from who is accountable for testing on any one project inside it? The sources this research read do not agree those are the same role; which is it here? Who else needs to read this document even though they do not own or execute against it? PRIORITY Name the document's own owner first. Every other row, including who executes testing on any one project, depends on knowing who actually approves a change to the document itself. ROW HINT A good row names a role and the actual person or team holding it, and states what that role does with respect to this document, not to any one project's own testing. A weak row names neither a holder nor a responsibility. GOOD | Program-level strategy owner | VP of Engineering, Granite Peak Utilities | Owns this document; approves changes (see Governance and Policy Alignment above) | WEAK | QA | | (no holder, no stated responsibility) TRAP Naming only the roles that execute testing on one project and omitting the person or group who owns this document itself across every project in scope. The sources disagree on whether those are the same role; this table should not quietly assume they are. -->
| Role | Holder | Responsibility ||---|---|---|| {{role_name}} | {{role_holder}} | {{role_responsibility}} |
## Review Trigger
<!-- WHAT Named conditions, not a calendar reminder, that tell this organization or program the strategy has gone stale, each with a named owner and what that owner does about it. WHY The standing-standards family this bundle belongs to requires every member to carry this mechanism, because every member fails the same quiet way: by drifting out of date while everyone still believes it is current. No source read for this bundle names who notices any of these conditions; the owner column is this family's own obligation, not a sourced fact. One vendor states plainly, "There is no universal review schedule." Where a source names an annual or semi-annual clause instead, that is one described governance model, not a measured norm across the field, and it belongs here only as a floor layered under a named condition, never as a calendar rewrite of this section's own rule. Candidate conditions this research supports: a project's own test plan records a deviation from this strategy, which is this strategy's own inheritance mechanism signalling a possible drift; the test policy this strategy aligns with changes; a test summary report shows a strategy-mandated level or type was skipped on a real release; or a new test level, tool, or environment is adopted across more than one project without this document being revised to say so. Deep dive: test-strategy_companion.md section 3 (Anatomy > Review Trigger). ASK What concrete, recurring condition would tell us this strategy is out of date: a recorded deviation, a changed policy, a skipped mandated level, or a tool or environment adopted without this document changing? Who actually notices each one, by name or by role? What do they do about it once they notice? PRIORITY List the deviation-recording condition first. It is this strategy's own built-in signal, surfaced by test-plan's own test_strategy_ref field and Risk-Ranked Approach section, so it needs no new instrumentation to start working. ROW HINT A good row names a condition someone could actually notice without being told to look, a named owner, and what that owner does about it. A weak row is a calendar date with no condition and no stated action. GOOD | A project's own test plan records a deviation from this strategy's levels, types, or tooling | Program QA Lead | Reviews whether the deviation reveals the strategy itself is now wrong, not just that one project's own case | WEAK | Reviewed annually | | (a calendar reminder; it fires whether or not anything is actually wrong, and names no owner's action) TRAP Writing only a calendar reminder. One vendor source states plainly that no universal review schedule exists; where an annual clause is genuinely useful, write it as a floor under a named condition in this table, never as a substitute for one. -->
| Condition | Owner | What They Do About It ||---|---|---|| {{review_trigger_condition}} | {{review_trigger_owner}} | {{review_trigger_action}} |---title: "Acme Analytics - Reporting Platform Modernization Test Strategy"doc_type: test-strategysize: leanorganization_or_program_name: "Acme Analytics - Reporting Platform Modernization"owner: "Anjali Rao (QA Lead, Reporting)"status: activedoc_version: "1.0.0"created: "2026-07-20"updated: "2026-07-20"related_links: - "../test-summary-report/test-summary-report_example.md (Saved Views summary report, final 2026-07-17; its DEF-2291 finding is why this strategy exists)" - "../test-plan/test-plan_example.md (Saved Views for Dashboards Test Plan, approved 2026-07-06, before this strategy existed; its test_strategy_ref field predates the seam this document now fills)"source_template: test-strategysource_template_version: 0.1.0---
> **Worked example.** A filled `test-strategy`, the lean variant this bundle ships in only one size, for> Acme Analytics' Reporting Platform Modernization program: the standing document the> [`test-plan`](../test-plan/test-plan_example.md) example's own `test_strategy_ref` field names as absent> ("Acme Reporting has no standing test strategy document") at that plan's date of 2026-07-06. It is dated> 2026-07-20, the same program steering review date the> [`kpi-dashboard`](../kpi-dashboard/kpi-dashboard_example.md) and> [`raid-log`](../raid-log/raid-log_example.md) examples were last reviewed, and three days after the> [`test-summary-report`](../test-summary-report/test-summary-report_example.md) example went final on> 2026-07-17 and drew the lesson behind DEF-2291: an entitlement assertion written against individual rows> cannot catch a leak in an aggregate computed across them, the gap every entitlement case written before> that release's own case review shared. Per the `standing-standards` family contract, this document's> chaining stays loose by design: a program agrees it once and consults> it repeatedly, rather than any one project's release producing it. No test plan in this library yet> names it in a `test_strategy_ref` field; this is that seam's first filled strategy. Read it alongside> [`test-strategy_guide.md`](test-strategy_guide.md), the rubric it was graded against. Names already> established in the library's Acme Analytics thread are reused as established; the Scheduled Exports> project, the data residency attestation, and every figure marked illustrative are this example's own> invention.
# Acme Analytics - Reporting Platform Modernization Test Strategy
## Purpose and Scope
Acme Analytics had never run a program-level test strategy until this one. The gap showed up directly:Saved Views for Dashboards, the Reporting Platform Modernization program's first project, caught DEF-2291only because a case review added an aggregate assertion to a High-tier case five days before that casefired. Every entitlement case written before that review had asserted onindividual rows, and every row was always correct; nothing told the team that an entitlement check has toassert on an aggregate as well as a row. The project's own test plan had no program default to inherit,only its own judgment, so the next project would have had to rediscover the same lesson on its own. Thisstrategy binds every project the Reporting Platform Modernization program runs under its own releasecalendar: today, that is the Reporting squad's ongoing Saved Views work, and the forthcoming ScheduledExports project the program has not yet staffed (invented for this example; no other document in thislibrary names it). A project signs on the same way `test-plan` already expects: it writes this strategy'sname into its own `test_strategy_ref` field, rather than drafting defaults from nothing.
**Explicitly out of scope:** The Growth pillar's own A/B experimentation platform, whose testing validatesstatistical experiment design rather than release readiness and will keep its own separate strategy onceone is written; Acme's data residency attestation with its cloud hosting provider, which that provider'sown audit process covers, not any project's test plan; and the legacy reporting flow that predates thisprogram, which stays under its existing regression suite until it is migrated onto the platform thisprogram builds.
## Test Levels and Types
| Level or Type | Default or Mandatory | Reason or Obligation ||---|---|---|| Unit and integration (CI gate) | Default | Every level below assumes the CI gate is already green; a project that skips it is testing on top of an unknown foundation || WCAG 2.2 AA accessibility testing | Mandatory | Written into Acme Analytics' enterprise customer contracts, not a government accessibility law; Legal confirmed this with Sales before this strategy was written || Security review of any change touching a permission boundary | Mandatory | A review by Security found the gap behind DEF-2291 before it shipped: an entitlement case that asserts only on rows cannot see a leak in an aggregate computed across them; this gate makes that review a standing requirement rather than each project's own discovery || Data tests on any new or changed table | Default | A project that ships a pipeline or warehouse table runs, at minimum, a uniqueness check, a not-null check, an accepted-values check and a referential check on it before the table goes live || Load testing of any new or materially changed query path | Default | Reporting workloads spike at month-end close; a new query path is smoke-tested first, then run at progressively higher load toward its expected ceiling before production || Contract testing across service boundaries | Default | The program's services talk to each other over internal APIs that can change independently; a consumer-driven contract test runs in CI before either side of a boundary ships a breaking change |
## Standard Tooling and Environments
Every project defaults to the same environment ladder Reporting already runs Saved Views through: aCI-only unit and integration run on every commit; a shared Staging tier, reseeded nightly, for functionaland end-to-end work; a Pre-prod replica, sized to match production load, for anything performancesensitive; and, for any project that changes stored data, a separate migration sandbox for dry runs androllback rehearsals before anything touches Staging. A project may add an environment Platform does notalready provide, a dedicated load-test harness, for instance, but it still passes through CI and Stagingfirst; it does not skip straight to a bespoke environment of its own.
**Rule for real or production-derived data in test:** Staging and the migration sandbox run on anonymizedor synthetic data by default; no project loads an unanonymized production extract into either one as amatter of course. A project that hits a case anonymized data genuinely cannot exercise, a defect that onlyreproduces against one real customer's data shape, say, opens a time-boxed exception instead of loadingreal data on the sly. The request names which tables are involved and goes to Dana Osei for anythingtouching the shared database; nothing loads until she signs off. The exception rides with the releasewindow it was opened for (illustrative: two to three weeks), and a later release starts the request overrather than inheriting it.
## Governance and Policy Alignment
No test policy exists above this strategy. Acme Analytics has never written one, and nothing here shouldbe read as implying one is coming soon. If Acme Analytics adopts a test policy later, aligning thisstrategy to it becomes the first item on the agenda of whichever steering review follows that adoption,ahead of whatever else that review was already going to cover.
**Who approves a change to this strategy:** Marta Reyes, Program Manager, at the program's monthlysteering review. A change that cannot wait for that cadence, correcting a wrong environment name, say, maybe made by Anjali Rao directly and reported at the next review rather than held for it.
## How Project Test Plans Inherit or Deviate
Adoption is simply writing this strategy's name into a project's own `test_strategy_ref` field; thatsingle step is the whole of it, which is why the field already existed in `test-plan` before this documentdid. Where a project's Risk-Ranked Approach departs from one of the defaults above, a skipped level, adifferent environment, a waived data test, the plan states the reason next to that row rather than in aseparate memo nobody reads later.
**Who may accept a deviation from this strategy:** Anjali Rao, as the strategy's own owner, accepts adeviation from any Default row once the stated reason is written down. A deviation touching a Mandatoryrow also needs Marta Reyes's sign-off: a Mandatory row exists because of an obligation this program doesnot get to waive on one QA Lead's say alone. A deviation with no reason written down is not accepted, andthe plan stays unapproved until one is.
## Roles and Responsibilities Across Projects
| Role | Holder | Responsibility ||---|---|---|| Strategy owner | Anjali Rao, QA Lead, Reporting | Owns this document, drafts every revision, and brings a proposed change to the program's steering review || Approver | Marta Reyes, Program Manager | Approves this document and its revisions at the program's monthly steering review || Shared-infrastructure liaison | Dana Osei, Staff Engineer, Platform | Confirms the environment and data-exception defaults above still match what Platform actually runs, and flags this document when they drift || Per-project execution | Each project's own test plan owner (Anjali Rao, for Saved Views, as of this writing) | Runs testing for their own project against these defaults, recording any deviation in their own plan |
## Review Trigger
| Condition | Owner | What They Do About It ||---|---|---|| A project's own plan states a deviation from one of this strategy's defaults | Anjali Rao | Checks whether the same deviation has now shown up in more than one project's plan, or whether it touches one of the Mandatory rows above; either one puts the default itself on the next steering review's agenda || Acme Analytics adopts a test policy above this strategy, or changes one it already has | Anjali Rao | Checks every section above against the new policy and updates whatever no longer aligns, ahead of the next scheduled steering review || A test summary report shows a mandatory level or type above was skipped on a shipped release | Marta Reyes | Brings the skip to the next steering review as a possible defaults gap, the same kind of gap a case review caught in DEF-2291 before it could ship rather than after || A new test level, tool or environment is adopted on more than one project without this document naming it as a default | Anjali Rao | Proposes adding it as a program default rather than letting each project keep rediscovering it on its own |
Acme Analytics has no audit or regulatory requirement forcing a calendar review of this document. Absentone, the conditions above are the whole of how this strategy stays current; there is no separate annualreview date standing in for them.Provenance
Section titled “Provenance”The reasoning, the history and every source, in the repository:
- Companion - the long-form argument: why these sections, where the sources disagree, and what the bundle refuses to claim
- History - what changed in this bundle, and when
- Research log - every source consulted, with what each one actually supports
- Catalog metadata - the machine-readable record this page is generated from
Catalog record: 7 sections across 1 format(s), methodology methodology-agnostic, typically owned by QA Manager.