Trade Study
beta · Family decision-docs · Phase develop · Sizes lean, full · ~4,450 tokens
Compares a named set of technical alternatives against weighted criteria and records the recommendation that comparison supports, for a problem the organization has already decided to solve. Evaluates options; it does not have to be where the resulting decision is recorded, which is an ADR’s job.
Fast reference for using the trade-study bundle. For the full reasoning, history, and sources, read
trade-study_companion.md. A fully worked instance is
trade-study_example.md.
When to use
Section titled “When to use”- You already own the problem, and now have to choose among named technical approaches to solve it: a runner model, a message-ingestion technology, a persistence engine, an architecture pattern.
- More than one approach is genuinely on the table, and the choice is not obvious from where you stand today.
- The comparison needs to survive a reader who was not in the room: a reviewer, an auditor, or whoever revisits this decision in a year and wants to know what the road not taken would have cost.
- Reasonable people on the team would weight the criteria differently, and writing the weights down, with a reason for each, is cheaper than arguing from memory later.
- You are about to feed an ADR, an RFC, or an SDD a weighted comparison, rather than a paragraph of reasoning, for their own Alternatives Considered prose.
When NOT to use
Section titled “When NOT to use”- There was never a real second option. Skip this document and write the decision straight into
whatever record normally follows one (see
adr); companion section 4 names the source that says to skip it. - You are deciding whether to invest in something at all, not how to solve a problem you already own. No definition this bundle’s research read frames a trade study as an investment decision; cost sits inside the comparison as one criterion among several, not as the question itself. That is a business case’s job, and it is this bundle’s own boundary to draw, stated plainly rather than borrowed from a source (see companion section 1 and section 8).
- The decision is already made and you want to justify it after the fact. A trade study with the answer chosen before the criteria is the predetermined-winner failure named below. Write the ADR honestly instead, or rerun the comparison for real.
- You want feedback on a proposal before anything is decided. That is an RFC’s job: it asks for input before the call is made. This document evaluates named alternatives against stated criteria; it does not gather opinions on whether to proceed.
- You are describing how a system will be built, not choosing among named approaches to building it. That is an SDD’s job. An SDD’s own Alternatives Considered section can cite a finished trade study instead of rebuilding the matrix in prose.
- The alternatives cannot yet be named with confidence. A spike report exists to resolve that uncertainty first. Run it before the Alternatives section below can be filled in honestly.
Trade study, RFC, ADR, or SDD?
Section titled “Trade study, RFC, ADR, or SDD?”| Trade study | RFC | ADR | SDD | |
|---|---|---|---|---|
| Answers | “Which named option wins, and why?” | “Should we, and which way?” | “What did we decide, and why?” | “How will we build it?” |
| Timing | Before the decision, once alternatives can be named | Before the decision | After the decision | Before or while building |
| Produces | A scored comparison and a recommendation | A proposal | A record | A description |
| Records the decision itself | No, it points to the record that does | No | Yes, immutably | No |
These are a division of labor, not four names for one job. A trade study’s Recommendation section may name the decision taken in one line; the ADR is where that decision is recorded for good.
Pick a variant
Section titled “Pick a variant”Lean (five sections, the default): Problem and Decision Framing, Criteria and Weights, Alternatives, Scoring Matrix, Recommendation. Write lean when the stakes are contained and the comparison is between a small set of options whose tradeoffs are already mostly understood. The sources allow a short report as readily as a formal one (companion section 4).
Full (nine sections): adds Sensitivity Analysis, Risk and Benefit, Dissent or Independent Review, and References. Reach for full when the decision is high-stakes or hard to reverse, the team disagrees about the criteria or their weights, or a reviewer outside the immediate group will need to check the reasoning later.
Grow lean into full by adding sections; never reorder or rename the shared ones, because the full variant is a strict superset of lean. The scaling signal is how hard the decision is to reverse and how contested the criteria are, not the size of the system being decided about.
One caution that applies to either variant. If the do-nothing row in Alternatives is also doing the job of a scoring reference point (a Pugh datum), say so explicitly, because a poor or biased reference option can distort the entire comparison. A do-nothing baseline and a Pugh datum are not automatically the same thing; using one as both is a choice the author makes, not a default the template assumes (companion section 6.3).
Quality rubric (self-grade before you ship it)
Section titled “Quality rubric (self-grade before you ship it)”Score each 0, 1, or 2. Under 14 out of 20 and the recommendation should not be acted on without a second look; send the document back for another pass before anyone treats it as settled.
Rows 7 through 10 grade sections the lean variant does not ship: Sensitivity Analysis, Risk and Benefit, Dissent or Independent Review, and References. Grade lean against the scope table below the main table instead of against all ten rows.
| # | Criterion | 0 | 1 | 2 |
|---|---|---|---|---|
| 1 | Question answered, not posed | No question stated, or the title names a topic rather than a comparison | A question is stated, but no decision maker is named or the decision it feeds is unclear | A reader can point at the sentence naming the question, the decision it feeds, and who decides it |
| 2 | Weights justified and typed | Weights appear with no stated reason | Each weight has a reason, but the document never says whether a weight means importance or the swing between the best and worst option | Every weight carries a stated reason, and the document names which of the two meanings its weights use |
| 3 | Do-nothing row scored | No do-nothing or do-minimum row, and no line explaining its absence | A do-nothing row exists, but it is a label with no real cost or risk behind it | The do-nothing row is scored as a genuine alternative, or a specific reason for its absence is written in Alternatives |
| 4 | Scores have a basis | Scores are bare numbers | Some scores carry a basis, others do not | Every score in the matrix carries a basis a reader could dispute, and the weighted total can be recomputed from the rows shown |
| 5 | Recommendation reasons from rows | The recommendation restates the highest weighted total with no reasoning | The recommendation names which rows it depends on, but states no condition under which it would change | A reader can name the specific rows the recommendation depends on and the condition that would overturn it, and a genuine near-tie is reported as one rather than forced apart |
| 6 | Record pointer given | No decision is named, and no record is pointed to | A decision is named, but no record is pointed to, or the pointer is to this same document | The line names the decision taken and a specific record elsewhere that holds it, or states plainly that no decision has been taken yet |
| 7 | Sensitivity tested (full) | No weight or score was moved; confidence is asserted rather than tested | A weight or score was moved, but the result is not stated either way | A specific weight or score was moved by a stated amount, and the document says plainly whether the recommendation survives |
| 8 | Every contender covered (full) | Only the recommended option has a risk or a benefit written down | More than one option is covered, but the mitigation the leading option would need is missing | Every alternative still in contention has a risk, a benefit, and the leading option’s mitigation is named |
| 9 | Reviewer independence stated (full) | The section is silent, or says only that the team reviewed its own work | A reviewer is named, but their stake in the outcome is not addressed | A named person or group checked the scoring, and the document states how independent of the outcome they are, or says plainly that no independent check happened |
| 10 | References dated, checkable (full) | Sources are named generically (“various documents”) or not at all | Specific sources are named, but without a date | Each criterion’s score traces to a named, dated source a reader could open and check |
Score lean against this table instead of the headline threshold above.
| Variant | Rows that apply | Maximum | Score against |
|---|---|---|---|
| full | all 10 | 20 | 14 |
| lean | 1-6 (it carries none of the four full-only sections) | 12 | 9 |
Every cell above describes evidence, not a count. A rubric cell that can be satisfied by adding items will be satisfied by adding items, so none of them ask for a number of anything; each asks whether a reader could point at the specific sentence that does the work.
Named anti-patterns (the usual wrecks)
Section titled “Named anti-patterns (the usual wrecks)”Each one is sourced in companion section 7.
- Criteria chosen after the alternatives, to fit the one already wanted. The list quietly favors the option someone preferred before scoring began. Set the criteria and their weights before anyone scores an option.
- A criterion with no way to measure it. A criterion only opinion can score gives a favored option somewhere to hide. Pick criteria that are transparent and measurable before scoring starts.
- A must-have outvoted by a pile of minor criteria. Weighting does not stop scoring from equalizing requirements, so enough small criteria can add up to select an option that fails something the team actually needed. Mark a must-have mandatory instead of only weighting it.
- A recommendation that survives no real test, because nothing was moved. If a minor change in one input changes the answer, the study is probably not sound, and a matrix with no sensitivity check has nothing to answer that criticism with.
- Scorers using the same scale differently, with no discussion to close the gap. One scorer uses the full range and another clusters in the middle. Left unaddressed, two scorers are not comparing the same thing.
- Averaging away a disagreement instead of resolving it. A quiet average, or a vote, hides exactly the disagreement a reader would want surfaced.
- A biased or unexamined datum, when the scoring method uses one. A poor reference option can distort the entire comparison. Naming the datum and testing whether the ranking survives a different one is the fix.
- Treating the weighted total as the decision rather than a prompt for one. The option with the highest score is not necessarily the one to choose. The total tells you where the discussion should focus, not what to conclude from it.
The artifacts
Section titled “The artifacts”trade-study_template-lean.md · ~4,450 tokens
---title: "{{title}}"status: "{{status}}"date: "{{date}}"decision_makers: ["{{decision_makers}}"]doc_type: trade-studysize: leansource_template: trade-studysource_template_version: 0.1.0---
<!--LEAN TRADE STUDY. The five sections this bundle treats as the minimum comparison: Problem andDecision Framing, Criteria and Weights, Alternatives, Scoring Matrix, and Recommendation. This isthe right default. A practical reading of every source that addresses formality points the sameway: write lean when the stakes are contained and the comparison is between a small set ofoptions whose tradeoffs are already mostly understood. To grow this into a full trade study, ADDsections (see trade-study_template-full.md); never rename or reorder the ones below, because thefull variant is a strict superset of this one. See trade-study_companion.md section 4 (Variantsand sizing).
Microsoft's own engineering playbook goes further and names the condition for skipping theexercise entirely: "Trade studies should be avoided if there is a clear solution choice." Whenthere is a clear design, it says, "the trade study should be omitted, and an entry should be madein the Decision Log documenting the decision." If there was never a real second option, writenothing and log the decision directly instead of filling this out for its own sake.
THIS IS NOT A BUSINESS CASE, AND THAT DRIFT IS THE EASIEST MISTAKE TO MAKE HERE.A trade study compares named technical approaches to a problem the organization has alreadydecided to solve; cost sits inside the comparison as one criterion among several, not as thequestion itself. No definition this bundle's research could read frames a trade study as thedecision whether to invest. If you are actually asking whether to invest in something at all,rather than how best to solve a problem you already own, write a business case instead. Seetrade-study_companion.md section 1 (Orientation) and section 8 (Relationships to otherartifacts).
THIS EVALUATES OPTIONS; IT DOES NOT RECORD THE DECISION.This bundle's own division of labor hands the decision itself to an ADR, written once therecommendation below is acted on. An RFC or an SDD whose own Alternatives Considered prose needsa weighted comparison can cite this document instead of rebuilding the matrix in prose. A spikereport, if one came first, reduced the uncertainty that let these alternatives be named withconfidence in the first place. See trade-study_companion.md section 8 (Relationships to otherartifacts).
THE DO-NOTHING BASELINE IS THIS BUNDLE'S OWN REQUIREMENT, NOT A DOCUMENTED CONSENSUS.NASA notes only that "(Sometimes doing nothing or not making a decision is an option.)" The FHWAis just as conditional: "It is often good practice to include a" ... "alternative as the basecase, depending on the nature of the trade study." Several other named outlines mention no suchrow at all. This bundle requires it anyway: addingor dropping a do-nothing row can change which live option ranks first, because that is exactlythe mechanism a rank reversal describes. A comparison with no credible do-nothing option shouldsay so in one line in Alternatives, rather than silently drop the row. Seetrade-study_companion.md section 3 (Anatomy > Alternatives, Including a Do-Nothing Baseline).
WHERE THIS FILE GOES: wherever your team keeps design and decision records. This bundleprescribes no location. Once a decision is taken from this comparison, link this document fromthe ADR that records it; do not fold the two together.
HOW TO FILL THIS IN1. Read the comment under each heading: WHAT it wants, WHY it matters (with a pointer into trade-study_companion.md for the deep reasoning), guiding questions to ASK, a GOOD and a WEAK example, and the TRAP to avoid. For a table, PRIORITY explains the coverage rule and ROW HINT says what a good row contains.2. Replace each {{placeholder}} with your content. If a section does not apply, write "N/A" and one line of why, rather than deleting it.3. Before you ship it: self-grade against trade-study_guide.md, then DELETE every HTML comment. They are guidance, not content.-->
# {{title}}
<!-- WHAT A title naming both the problem this study is solving and, once you know it, the option it recommends. A statement, not a topic. WHY A comparison filed under a topic name alone, such as CI runner options, tells a later reader nothing about what was decided; a title naming the recommendation lets them judge in one line whether to open the rest. Deep dive: trade-study_companion.md section 3 (Anatomy > Problem and Decision Framing). ASK What problem does this comparison solve? Once you have an answer below, does the title still read true, or does it only describe the question? GOOD "CI runner model for the platform team: recommend self-hosted over the hosted service" WEAK "CI options" (a topic, not a comparison; nobody can tell what was found) TRAP Titling the technology area instead of the comparison. A title can be written before you start and revised once you finish; revise it, rather than shipping the question as if it were the answer. -->
## Problem and Decision Framing
<!-- WHAT The question this study answers, the decision it feeds and who decides it, and the constraints and assumptions every alternative below must respect. WHY The FHWA puts this first for a reason: "First, define the question the trade study is to answer." Everything below is only as good as the question it is answering, and drift here is where the predetermined-winner failure gets its opening, when criteria get written to fit an answer already chosen rather than the other way around. Deep dive: trade-study_companion.md section 3 (Anatomy > Problem and Decision Framing), section 6.7 (The predetermined-winner critique). ASK What question is this study actually answering? What decision does the answer feed, and who makes that decision? What constraint or assumption must every alternative respect, whether or not anyone ever says it out loud? GOOD "Question: should the platform team keep the current shared CI runner pool, or move to a different model, before the fourth product team onboards next quarter. Decision this feeds: which runner model to standardize on for the next two years. Decided by: the platform lead, with sign-off from the three onboarding teams' leads. Constraints: must support the existing container build images without a rewrite; budget ceiling of 4,000 dollars a month." WEAK "We need a CI solution." (no question evidence could answer, no named decision maker, no constraint) TRAP Writing this section after the alternatives are already chosen, to justify them. A question and a decision maker written down before the comparison starts is what keeps the criteria in the next section from being reverse-engineered to fit an answer you already picked. -->
* Question this study answers: {{the_question}}* Decision this feeds, and who decides: {{decision_and_decision_makers}}* Constraints and assumptions every alternative must respect: {{constraints_and_assumptions}}
## Criteria and Weights
<!-- WHAT Each criterion this study will score alternatives against, its weight, why it was chosen and weighted that way, whether it is mandatory, and the scoring method and scale in use. WHY The FHWA's sequencing exists on purpose: criteria are set "before the alternatives are selected, to avoid subconsciously biasing the criteria to favor a pre-determined preferred alternative." ECSS requires the weighting itself to be justified, not merely stated. A weight also carries one of two different meanings, importance or how much the best and worst alternative actually differ (a swing weight), and this template asks you to say which one you mean rather than picking one for you. Deep dive: trade-study_companion.md section 3 (Anatomy > Criteria and Weights), section 6.2 (A separate must-have screen, or one weighted pass), section 6.4 (One weight, two meanings). ASK What criteria actually matter here, and why these and not others? Is each weight measuring importance, or the swing between the best and worst alternative on that criterion, and have you said which? Is any criterion mandatory, meaning an alternative that fails it is dropped outright rather than merely docked points? PRIORITY Set every criterion and its weight before you look hard at the alternatives in the next section, to avoid biasing the criteria toward an answer you have already chosen. Mark a criterion mandatory only when failing it should drop the alternative outright, not merely cost it points. ROW HINT A good row states the weight and the reason for that number, not the number alone. "Reliability, weight 30" with no reason cannot be checked six months later; "weight 30, because an outage here is customer-visible within minutes" can. GOOD | C1 | Queue wait at peak load | 30 | No | Weighted this high because the fourth team's onboarding is what forces this decision, and the shared pool already hits its job cap at peak | WEAK | C1 | Good support | High | | Because it matters | TRAP Picking the criteria after seeing how each alternative would score, so the list quietly favors the one you already wanted. If a criterion only occurred to you after looking at the options, say so and weigh honestly whether it belongs. -->
Scoring method and scale in use: {{scoring_method_and_scale}}What a weight means here (importance, or the swing between the best and worst alternative on thatcriterion): {{weight_meaning}}
| ID | Criterion | Weight | Mandatory? | Justification ||---|---|---|---|---|| {{criterion_id}} | {{criterion_name}} | {{weight}} | {{mandatory_yes_no}} | {{criterion_justification}} |
## Alternatives, Including a Do-Nothing Baseline
<!-- WHAT Every alternative actually considered, including a do-nothing or do-minimum baseline, and, if a larger field was narrowed first, how. WHY The FHWA's own advice: "Be sure that all reasonable alternatives are on the table." NASA's trade tree is the named tool for narrowing a large field before detailed analysis begins. The do-nothing row is this bundle's own requirement, not a documented consensus; see the note at the top of this file and the reasoning behind it. Deep dive: trade-study_companion.md section 3 (Anatomy > Alternatives, Including a Do-Nothing Baseline), section 6.3 (The Pugh datum and the do-nothing baseline are not the same thing). ASK What alternatives were genuinely on the table, not just the one you expect to win? If the field started larger, how was it narrowed, and on what basis? What would doing nothing, or the minimum, actually look like here, and what would it cost? PRIORITY Every comparison needs a do-nothing or do-minimum row, even when it is rejected in one line. If a comparison genuinely has no credible do-nothing option, write one line here saying why rather than silently dropping the row. ROW HINT A good row names a real alternative a reasonable team could actually choose, with what it would concretely involve. A weak row is a label with nothing behind it, or a straw alternative built to lose. GOOD | A1 | Keep the current shared runner pool (do-nothing) | No new cost, but queue time grows once the fourth team onboards | Yes | WEAK | A3 | The obvious choice | | No | TRAP Treating this table's do-nothing row as the same thing as a scoring reference point (a Pugh datum), if you are also using one. A datum is what every score is measured against; a do-nothing row is a candidate being scored in its own right. Say explicitly if the same alternative is doing both jobs. -->
How the field was narrowed to the alternatives below (or "N/A, nothing was narrowed"):{{how_the_field_was_narrowed}}
| ID | Alternative | Description | Do-nothing or do-minimum baseline? ||---|---|---|---|| {{alternative_id}} | {{alternative_name}} | {{alternative_description}} | {{is_do_nothing_baseline}} |
## Scoring Matrix
<!-- WHAT The completed comparison: each criterion against each alternative, a weighted total per alternative, and the basis for each individual score, laid out so a reader outside this team could still follow it. WHY The FHWA's orientation is this template's default, alternatives on one axis, criteria on the other, with a readability requirement stated directly: the matrix "should be easily comprehensible to management and other stakeholders". ASQ gives the arithmetic and the caution in one breath: the highest score "will not necessarily be the one to choose, but the relative scores can generate meaningful discussion". Deep dive: trade-study_companion.md section 3 (Anatomy > Scoring Matrix), section 6.1 (Is weighted scoring reliable, or only as reliable as its inputs). ASK What is the basis for each score, in one phrase a reader could check or dispute? Does each weighted total actually equal score times weight, summed across criteria? If two scorers disagreed on a cell, did you discuss it to a real answer, or quietly average the disagreement away? PRIORITY Every criterion from Criteria and Weights gets exactly one row here; every alternative from the table above gets its own column. Add a column for each further alternative and a row for each further criterion. ROW HINT A good row scores every alternative against one named criterion and gives the basis for each score beside the number. A row of bare numbers with no basis cannot be checked, disputed, or trusted later. GOOD "Queue wait at peak load, weight 30: self-hosted runners score 8, because the team can add machines as the fourth team's builds arrive; the hosted service scores 4, because its plan caps concurrent jobs at the level the shared pool already reaches at peak." WEAK "Queue wait: 8 / 4." (numbers with no basis; nothing here can be checked or disputed) TRAP Treating the highest weighted total as the decision rather than a prompt for one. The total tells you where the discussion should focus, not what to conclude from it; write Recommendation as reasoning from specific rows below, not as a restatement of this total. -->
| Criterion | Weight | {{alternative_1_name}} | {{alternative_2_name}} ||---|---|---|---|| {{criterion_name}} | {{weight}} | {{score}} ({{basis}}) | {{score}} ({{basis}}) || **Weighted total** | | {{weighted_total_1}} | {{weighted_total_2}} |
## Recommendation
<!-- WHAT The option the matrix supports, the rows it depends on, the conditions under which it holds, a genuine near-tie reported as one outcome rather than forced apart, and the decision actually taken, in one line, with a pointer to the record that holds it for good. WHY The FHWA's own instruction is plain: "Select and document the preferred candidate". ECSS requires both the choice and its conditions. NASA and Microsoft both allow a genuine near-tie, or a list of open questions, rather than forcing a false precision onto a comparison that cannot support it. This bundle's own division of labor stops the recommendation short of the permanent record, which belongs to an ADR. Deep dive: trade-study_companion.md section 3 (Anatomy > Recommendation), section 6.5 (One recommended option, or every closely ranked one), section 8 (Relationships to other artifacts). ASK Which option does the comparison actually support? Which specific rows does that depend on? Under what condition would this recommendation no longer hold? If two options are genuinely close, say that plainly, rather than reporting a winner the matrix cannot support. What decision was actually taken, and where is it recorded? GOOD "Recommended: the self-hosted runner, ahead by 4 points on the current weights, conditional on queue wait staying the team's top priority. Re-run this comparison if the hosted plan raises its concurrent-job cap; queue wait was the deciding criterion. Decision taken: adopt the self-hosted runner, recorded in ADR 0091." WEAK "We recommend the self-hosted option because it scored highest." (restates the matrix total, names no condition, points to no record) TRAP Forcing a clean winner out of a genuine near-tie. If two alternatives are within noise of each other on the weights you actually trust, say that plainly and name what would separate them, rather than dressing up a coin flip as a confident recommendation. -->
* Recommended alternative, or alternatives if genuinely close: {{recommended_option}}* Rows this depends on: {{dependent_rows}}* Conditions under which this holds: {{conditions}}* Decision taken, and where it is recorded for good: {{decision_and_record_pointer}}trade-study_template-full.md · ~6,300 tokens
---title: "{{title}}"status: "{{status}}"date: "{{date}}"decision_makers: ["{{decision_makers}}"]doc_type: trade-studysize: fullsource_template: trade-studysource_template_version: 0.1.0---
<!--FULL TRADE STUDY. All nine sections this bundle defines: the lean five, plus the four thisvariant adds, Sensitivity Analysis, Risk and Benefit, Dissent or Independent Review, andReferences. Use this weight when the decision is hard to reverse, the field disagrees about thecriteria or their weights, or a reviewer outside the immediate team will need to check thereasoning later. NASA allows "a simple white paper or a more formally formatted document",leaving the choice to "the mission and system context and the decision made". AcqNotes ties theheavier process to the stakes: "formal trade studies are indicated for high-value, high-risk, orother high-impact decisions." See trade-study_companion.md section 4 (Variants and sizing).
Most comparisons do not earn this weight. The lean variant (trade-study_template-lean.md) is thedefault, and Microsoft's own engineering playbook names the condition for skipping the exerciseentirely: "Trade studies should be avoided if there is a clear solution choice." The full variantis a strict superset of lean: the shared sections keep their names and their order, and this fileonly ADDS.
THIS IS NOT A BUSINESS CASE, AND THAT DRIFT IS THE EASIEST MISTAKE TO MAKE HERE.A trade study compares named technical approaches to a problem the organization has alreadydecided to solve; cost sits inside the comparison as one criterion among several, not as thequestion itself. No definition this bundle's research could read frames a trade study as thedecision whether to invest. If you are actually asking whether to invest in something at all,rather than how best to solve a problem you already own, write a business case instead. Seetrade-study_companion.md section 1 (Orientation) and section 8 (Relationships to otherartifacts).
THIS EVALUATES OPTIONS; IT DOES NOT RECORD THE DECISION.This bundle's own division of labor hands the decision itself to an ADR, written once therecommendation below is acted on. An RFC or an SDD whose own Alternatives Considered prose needsa weighted comparison can cite this document instead of rebuilding the matrix in prose. A spikereport, if one came first, reduced the uncertainty that let these alternatives be named withconfidence in the first place. See trade-study_companion.md section 8 (Relationships to otherartifacts).
THE DO-NOTHING BASELINE IS THIS BUNDLE'S OWN REQUIREMENT, NOT A DOCUMENTED CONSENSUS.NASA notes only that "(Sometimes doing nothing or not making a decision is an option.)" The FHWAis just as conditional: "It is often good practice to include a" ... "alternative as the basecase, depending on the nature of the trade study." Several other named outlines mention no suchrow at all. This bundle requires it anyway: addingor dropping a do-nothing row can change which live option ranks first, because that is exactlythe mechanism a rank reversal describes. A comparison with no credible do-nothing option shouldsay so in one line in Alternatives, rather than silently drop the row. Seetrade-study_companion.md section 3 (Anatomy > Alternatives, Including a Do-Nothing Baseline).
WHERE THIS FILE GOES: wherever your team keeps design and decision records. This bundleprescribes no location. Once a decision is taken from this comparison, link this document fromthe ADR that records it; do not fold the two together.
HOW TO FILL THIS IN1. Read the comment under each heading: WHAT it wants, WHY it matters (with a pointer into trade-study_companion.md for the deep reasoning), guiding questions to ASK, a GOOD and a WEAK example, and the TRAP to avoid. For a table, PRIORITY explains the coverage rule and ROW HINT says what a good row contains.2. Replace each {{placeholder}} with your content. If a section does not apply, write "N/A" and one line of why, rather than deleting it.3. Before you ship it: self-grade against trade-study_guide.md, then DELETE every HTML comment. They are guidance, not content.-->
# {{title}}
<!-- WHAT A title naming both the problem this study is solving and, once you know it, the option it recommends. A statement, not a topic. WHY A comparison filed under a topic name alone, such as CI runner options, tells a later reader nothing about what was decided; a title naming the recommendation lets them judge in one line whether to open the rest. Deep dive: trade-study_companion.md section 3 (Anatomy > Problem and Decision Framing). ASK What problem does this comparison solve? Once you have an answer below, does the title still read true, or does it only describe the question? GOOD "CI runner model for the platform team: recommend self-hosted over the hosted service" WEAK "CI options" (a topic, not a comparison; nobody can tell what was found) TRAP Titling the technology area instead of the comparison. A title can be written before you start and revised once you finish; revise it, rather than shipping the question as if it were the answer. -->
## Problem and Decision Framing
<!-- WHAT The question this study answers, the decision it feeds and who decides it, and the constraints and assumptions every alternative below must respect. WHY The FHWA puts this first for a reason: "First, define the question the trade study is to answer." Everything below is only as good as the question it is answering, and drift here is where the predetermined-winner failure gets its opening, when criteria get written to fit an answer already chosen rather than the other way around. Deep dive: trade-study_companion.md section 3 (Anatomy > Problem and Decision Framing), section 6.7 (The predetermined-winner critique). ASK What question is this study actually answering? What decision does the answer feed, and who makes that decision? What constraint or assumption must every alternative respect, whether or not anyone ever says it out loud? GOOD "Question: should the platform team keep the current shared CI runner pool, or move to a different model, before the fourth product team onboards next quarter. Decision this feeds: which runner model to standardize on for the next two years. Decided by: the platform lead, with sign-off from the three onboarding teams' leads. Constraints: must support the existing container build images without a rewrite; budget ceiling of 4,000 dollars a month." WEAK "We need a CI solution." (no question evidence could answer, no named decision maker, no constraint) TRAP Writing this section after the alternatives are already chosen, to justify them. A question and a decision maker written down before the comparison starts is what keeps the criteria in the next section from being reverse-engineered to fit an answer you already picked. -->
* Question this study answers: {{the_question}}* Decision this feeds, and who decides: {{decision_and_decision_makers}}* Constraints and assumptions every alternative must respect: {{constraints_and_assumptions}}
## Criteria and Weights
<!-- WHAT Each criterion this study will score alternatives against, its weight, why it was chosen and weighted that way, whether it is mandatory, and the scoring method and scale in use. WHY The FHWA's sequencing exists on purpose: criteria are set "before the alternatives are selected, to avoid subconsciously biasing the criteria to favor a pre-determined preferred alternative." ECSS requires the weighting itself to be justified, not merely stated. A weight also carries one of two different meanings, importance or how much the best and worst alternative actually differ (a swing weight), and this template asks you to say which one you mean rather than picking one for you. Deep dive: trade-study_companion.md section 3 (Anatomy > Criteria and Weights), section 6.2 (A separate must-have screen, or one weighted pass), section 6.4 (One weight, two meanings). ASK What criteria actually matter here, and why these and not others? Is each weight measuring importance, or the swing between the best and worst alternative on that criterion, and have you said which? Is any criterion mandatory, meaning an alternative that fails it is dropped outright rather than merely docked points? PRIORITY Set every criterion and its weight before you look hard at the alternatives in the next section, to avoid biasing the criteria toward an answer you have already chosen. Mark a criterion mandatory only when failing it should drop the alternative outright, not merely cost it points. ROW HINT A good row states the weight and the reason for that number, not the number alone. "Reliability, weight 30" with no reason cannot be checked six months later; "weight 30, because an outage here is customer-visible within minutes" can. GOOD | C1 | Queue wait at peak load | 30 | No | Weighted this high because the fourth team's onboarding is what forces this decision, and the shared pool already hits its job cap at peak | WEAK | C1 | Good support | High | | Because it matters | TRAP Picking the criteria after seeing how each alternative would score, so the list quietly favors the one you already wanted. If a criterion only occurred to you after looking at the options, say so and weigh honestly whether it belongs. -->
Scoring method and scale in use: {{scoring_method_and_scale}}What a weight means here (importance, or the swing between the best and worst alternative on thatcriterion): {{weight_meaning}}
| ID | Criterion | Weight | Mandatory? | Justification ||---|---|---|---|---|| {{criterion_id}} | {{criterion_name}} | {{weight}} | {{mandatory_yes_no}} | {{criterion_justification}} |
## Alternatives, Including a Do-Nothing Baseline
<!-- WHAT Every alternative actually considered, including a do-nothing or do-minimum baseline, and, if a larger field was narrowed first, how. WHY The FHWA's own advice: "Be sure that all reasonable alternatives are on the table." NASA's trade tree is the named tool for narrowing a large field before detailed analysis begins. The do-nothing row is this bundle's own requirement, not a documented consensus; see the note at the top of this file and the reasoning behind it. Deep dive: trade-study_companion.md section 3 (Anatomy > Alternatives, Including a Do-Nothing Baseline), section 6.3 (The Pugh datum and the do-nothing baseline are not the same thing). ASK What alternatives were genuinely on the table, not just the one you expect to win? If the field started larger, how was it narrowed, and on what basis? What would doing nothing, or the minimum, actually look like here, and what would it cost? PRIORITY Every comparison needs a do-nothing or do-minimum row, even when it is rejected in one line. If a comparison genuinely has no credible do-nothing option, write one line here saying why rather than silently dropping the row. ROW HINT A good row names a real alternative a reasonable team could actually choose, with what it would concretely involve. A weak row is a label with nothing behind it, or a straw alternative built to lose. GOOD | A1 | Keep the current shared runner pool (do-nothing) | No new cost, but queue time grows once the fourth team onboards | Yes | WEAK | A3 | The obvious choice | | No | TRAP Treating this table's do-nothing row as the same thing as a scoring reference point (a Pugh datum), if you are also using one. A datum is what every score is measured against; a do-nothing row is a candidate being scored in its own right. Say explicitly if the same alternative is doing both jobs. -->
How the field was narrowed to the alternatives below (or "N/A, nothing was narrowed"):{{how_the_field_was_narrowed}}
| ID | Alternative | Description | Do-nothing or do-minimum baseline? ||---|---|---|---|| {{alternative_id}} | {{alternative_name}} | {{alternative_description}} | {{is_do_nothing_baseline}} |
## Scoring Matrix
<!-- WHAT The completed comparison: each criterion against each alternative, a weighted total per alternative, and the basis for each individual score, laid out so a reader outside this team could still follow it. WHY The FHWA's orientation is this template's default, alternatives on one axis, criteria on the other, with a readability requirement stated directly: the matrix "should be easily comprehensible to management and other stakeholders". ASQ gives the arithmetic and the caution in one breath: the highest score "will not necessarily be the one to choose, but the relative scores can generate meaningful discussion". Deep dive: trade-study_companion.md section 3 (Anatomy > Scoring Matrix), section 6.1 (Is weighted scoring reliable, or only as reliable as its inputs). ASK What is the basis for each score, in one phrase a reader could check or dispute? Does each weighted total actually equal score times weight, summed across criteria? If two scorers disagreed on a cell, did you discuss it to a real answer, or quietly average the disagreement away? PRIORITY Every criterion from Criteria and Weights gets exactly one row here; every alternative from the table above gets its own column. Add a column for each further alternative and a row for each further criterion. ROW HINT A good row scores every alternative against one named criterion and gives the basis for each score beside the number. A row of bare numbers with no basis cannot be checked, disputed, or trusted later. GOOD "Queue wait at peak load, weight 30: self-hosted runners score 8, because the team can add machines as the fourth team's builds arrive; the hosted service scores 4, because its plan caps concurrent jobs at the level the shared pool already reaches at peak." WEAK "Queue wait: 8 / 4." (numbers with no basis; nothing here can be checked or disputed) TRAP Treating the highest weighted total as the decision rather than a prompt for one. The total tells you where the discussion should focus, not what to conclude from it; write Recommendation as reasoning from specific rows below, not as a restatement of this total. -->
| Criterion | Weight | {{alternative_1_name}} | {{alternative_2_name}} ||---|---|---|---|| {{criterion_name}} | {{weight}} | {{score}} ({{basis}}) | {{score}} ({{basis}}) || **Weighted total** | | {{weighted_total_1}} | {{weighted_total_2}} |
## Sensitivity Analysis
<!-- WHAT The one or two weights or scores you are least sure of, moved to see whether the recommendation survives, and the honest answer either way. WHY DSMC states the stakes directly: "If the solution can be changed with relatively minor changes in data input, the study is probably invalid". This is also the section's honest answer to the false-precision critique of the whole method; a matrix with no sensitivity check invites that critique and has no defense against it. Deep dive: trade-study_companion.md section 3 (Anatomy > Sensitivity Analysis), section 6.1 (Is weighted scoring reliable, or only as reliable as its inputs). ASK Which weight or score are you least confident in? How far would it have to move before a different alternative wins? Does the recommendation survive that move? If it does not, what does that imply about how confidently you can recommend anything? GOOD "Moved the queue-wait weight from 30 to 22 of the total 100 points, the number the team disagreed about most. The self-hosted runner still ranks first, by under a point. At 20 the two tie, and moving it to 15 flips the result to the hosted service. The recommendation holds for any queue-wait weight above 20." WEAK "We're fairly confident in our weights." (no variable named, nothing moved, nothing a reader can check) TRAP Running this check after the recommendation is already written, to defend it rather than test it. If nothing you tried could have changed the outcome, say exactly which values you moved and by how much, so a reader can see that for themselves rather than take your word for it. -->
* Weight or score tested: {{sensitivity_variable}}* How far it would have to move before the recommendation changes: {{sensitivity_threshold}}* Does the recommendation survive this test: {{recommendation_survives}}
## Risk and Benefit
<!-- WHAT The risks and benefits of every alternative still in contention, not only the one Scoring Matrix favors, plus the mitigation the leading alternative would need. WHY NASA's own decision report gives this content a row covering every option still under consideration. This bundle follows that wider scope rather than ECSS's narrower one, which scopes the same content to the recommended solution alone, because a reader comparing two close alternatives wants to see what each one would have cost, not only what the winner costs. That is this bundle's own choice between two sourced positions. Deep dive: trade-study_companion.md section 3 (Anatomy > Risk and Benefit). ASK What could go wrong with this alternative, and what would it cost if it did? What does this alternative buy that the others do not? For the alternative you expect to recommend, what would it take to live with its risk? PRIORITY Cover every alternative still in contention, not only the one Scoring Matrix favors. A reader comparing two close alternatives wants to know what each would have cost, not only what the winner costs. ROW HINT A good row names a risk or benefit specific to that alternative, with what it would take to live with it. "Could fail" or "has upside" said about every row equally is not a row, it is a shrug. GOOD | Self-hosted runners | Risk: a single maintainer leaving leaves nobody who knows the setup | Benefit: no per-minute vendor billing | Mitigation if recommended: document the setup and cross-train a second engineer before go-live | WEAK | Self-hosted runners | Risk: things could break | Benefit: it's ours | | TRAP Writing this section only for the alternative you are about to recommend. The risk and benefit of the road not taken is exactly what a reader revisiting this decision later will want to know. -->
| Alternative | Risk | Benefit | Mitigation (if this becomes the recommendation) ||---|---|---|---|| {{alternative_name}} | {{risk}} | {{benefit}} | {{mitigation_if_recommended}} |
## Recommendation
<!-- WHAT The option the matrix supports, the rows it depends on, the conditions under which it holds, a genuine near-tie reported as one outcome rather than forced apart, and the decision actually taken, in one line, with a pointer to the record that holds it for good. WHY The FHWA's own instruction is plain: "Select and document the preferred candidate". ECSS requires both the choice and its conditions. NASA and Microsoft both allow a genuine near-tie, or a list of open questions, rather than forcing a false precision onto a comparison that cannot support it. This bundle's own division of labor stops the recommendation short of the permanent record, which belongs to an ADR. Deep dive: trade-study_companion.md section 3 (Anatomy > Recommendation), section 6.5 (One recommended option, or every closely ranked one), section 8 (Relationships to other artifacts). ASK Which option does the comparison actually support? Which specific rows does that depend on? Under what condition would this recommendation no longer hold? If two options are genuinely close, say that plainly, rather than reporting a winner the matrix cannot support. What decision was actually taken, and where is it recorded? GOOD "Recommended: the self-hosted runner, ahead by 4 points on the current weights, conditional on the queue-wait weight staying above 20 (see Sensitivity Analysis). Re-run this comparison if the hosted plan raises its concurrent-job cap; queue wait was the deciding criterion. Decision taken: adopt the self-hosted runner, recorded in ADR 0091." WEAK "We recommend the self-hosted option because it scored highest." (restates the matrix total, names no condition, points to no record) TRAP Forcing a clean winner out of a genuine near-tie. If two alternatives are within noise of each other on the weights you actually trust, say that plainly and name what would separate them, rather than dressing up a coin flip as a confident recommendation. -->
* Recommended alternative, or alternatives if genuinely close: {{recommended_option}}* Rows this depends on: {{dependent_rows}}* Conditions under which this holds: {{conditions}}* Decision taken, and where it is recorded for good: {{decision_and_record_pointer}}
## Dissent or Independent Review
<!-- WHAT Who disagreed with the recommendation and how that was addressed, or who checked the scoring and how independent of the outcome they were. WHY SEBoK sets the strongest named standard: independent review "should come from an outside body" that "must be both organizationally and financially independent of the program in question." The FHWA's own review checklist asks the question directly: "Have key stakeholders reviewed and support the trade study results?" Deep dive: trade-study_companion.md section 3 (Anatomy > Dissent or Independent Review). ASK Did anyone on the team, or reviewing it, disagree with the recommendation? How was that disagreement addressed, not just noted? Who checked the scoring, and what is their stake in the outcome? GOOD "The engineer who favored the hosted service reviewed the final matrix and disputed its reliability score, not the weights; the team re-scored that row from the internal incident log, the recommendation held, and she accepted it. No reviewer outside the team checked the scoring independently." WEAK "The team reviewed its own work and is confident in the result." (answers a different question than whether anyone independent of the outcome checked it) TRAP Treating silence as agreement. A team that never surfaced a disagreement either genuinely had none, worth stating plainly, or suppressed one, which this section exists to catch either way. -->
* Who disagreed, and how it was addressed: {{dissent_and_resolution}}* Who checked the scoring, and how independent of the outcome they are: {{reviewer_independence}}
## References
<!-- WHAT The data, models, and documents the criteria, weights, and scores in this study actually rest on. WHY NASA's own report lists "all data sources used" among a trade study report's required content, and its decision report carries a dedicated row for it. ECSS's Annex L opens with applicable and reference documents as its second clause. Deep dive: trade-study_companion.md section 3 (Anatomy > References). ASK What document, benchmark, vendor page, or prior study did a specific score or weight actually come from? Is each one dated, since most of what a trade study cites changes with the product or the market it describes? GOOD "[1] Vendor X runner pricing page, read 2026-09-30, source for the job cap and the cost row. [2] Internal incident log, Q2 to Q3 2026, source for the reliability scores on both options." WEAK "Various internal documents and vendor sites." (names nothing a reader could open and check) TRAP Citing a product's current documentation as a timeless fact about the technology rather than a snapshot dated the day it was read. Date every technical fact a criterion's score actually rests on. -->
* {{reference_1}}* {{reference_2}}---title: "GPS telemetry ingestion for the truck fleet: recommend Amazon MSK over RabbitMQ Streams"status: "final"date: "2026-10-09"decision_makers: ["Marta Lindqvist, Platform Engineering Lead"]doc_type: trade-studysize: fullsource_template: trade-studysource_template_version: 0.1.0---
> **Worked example.** A filled `trade-study` for a fictional freight company, working name Thornwell> Freight, choosing a message-ingestion technology for GPS telemetry from its truck fleet. The full> variant is shown because the choice is hard to reverse once a customer contract depends on it and> the two engineers who would run it disagree about one of the weights. Every company name, person> name, figure, weight, score and date below is illustrative; where a technical claim about Kafka,> Amazon MSK or RabbitMQ Streams appears, it is a fact about that product as documented, not an> invented one.>> It is deliberately independent of the [`adr`](../adr/adr_example.md), [`rfc`](../rfc/rfc_example.md)> and [`sdd`](../sdd/sdd_example.md) examples rather than chained to them, and of> [`spike-report_example.md`](../spike-report/spike-report_example.md): the `decision-docs` family> teaches the difference between investigating a question, proposing an answer, evaluating named> options, recording a decision and describing a design, so each member's example is a different piece> of work. Read it alongside [`trade-study_guide.md`](trade-study_guide.md), the rubric it was graded> against.
# GPS telemetry ingestion for the truck fleet: recommend Amazon MSK over RabbitMQ Streams
## Problem and Decision Framing
* Question this study answers: the fleet-management platform at Thornwell Freight currently uploads each truck's full day of GPS positions once, overnight, when the truck docks at a depot and connects to depot Wi-Fi. A new customer contract requires geofence-exit alerts to reach dispatch within the hour of the event, not the next morning. The question is which message-ingestion technology should sit between the truck-side device and the dispatch dashboard to carry that continuous stream.* Decision this feeds, and who decides: whether to introduce a continuous ingestion layer at all, and if so, which technology to run it on, for the next two to three years. Decided by Marta Lindqvist, Platform Engineering Lead, with Sofia Reyes, Operations Lead, consulted because she owns the on-call rotation that will carry whichever system is chosen.* Constraints and assumptions every alternative must respect: the truck-side GPS device and its firmware are out of scope: a fleet-wide hardware change is not budgeted this cycle, and every alternative below assumes the device keeps sending the same message format it already sends. The new ingestion path must be able to run alongside the nightly batch during cutover rather than requiring a single flag day. The platform team is two engineers who also carry other production responsibilities, so whatever is chosen has to fit inside their existing on-call capacity. Whatever is chosen must be running before the new contract's go-live date, three months from this study.
## Criteria and Weights
Scoring method and scale in use: each alternative is scored 0 to 10 on each criterion below. Aweighted total is score times weight divided by 10, summed across the five criteria, so the maximumpossible weighted total is 100.What a weight means here: importance. None of the weights below are set to the swing between the bestand worst alternative on that criterion; the team discussed swing weighting and chose importancebecause two of the five criteria (cost and latency) do not yet have a wide enough spread betweenalternatives to weight by swing honestly.
| ID | Criterion | Weight | Mandatory? | Justification ||---|---|---|---|---|| C1 | Replicates every message to more than one node before acknowledging receipt | 15 | Yes | Marked mandatory because a truck's telemetry that exists on only one node is one disk failure away from vanishing outright, which Operations will not accept from anything meant to replace the nightly batch. Scored rather than left unweighted, so that the degree of exposure among the alternatives that do pass it still counts. || C2 | Operational burden on the two-person on-call rotation | 35 | No | Set to the highest weight here because neither platform engineer has run a distributed message broker in production, and whichever option is chosen has to be carried by two people without a dedicated hire. || C3 | Integration effort with the existing fleet-management pipeline | 25 | No | Weighted this high because the dispatch dashboard, the device-side agent and the existing data-warehouse job all read today's nightly-batch format, and every alternative below changes some part of that chain. || C4 | Cost at the fleet's current and projected message volume | 15 | No | Weighted lower than operational burden because the contract that forces this decision is worth more than any plausible difference in ingestion cost; every score on this row is this team's own illustrative estimate, not a vendor quote. || C5 | Latency from a truck leaving a geofence to the alert reaching the dispatch dashboard | 10 | No | Weighted lowest of the four non-mandatory rows because the new contract's own service level is "within the hour," which every live alternative clears comfortably; finer differences between them matter less than the other four rows. |
## Alternatives, Including a Do-Nothing Baseline
How the field was narrowed to the alternatives below: six named technologies were on the table beforenarrowing: self-managed Apache Kafka, Amazon MSK, Amazon Kinesis Data Streams, RabbitMQ Streams, NATSJetStream and Google Cloud Pub/Sub. Kinesis Data Streams was set aside first, on a procurementconstraint rather than a technical one: Thornwell's existing data-warehouse contract already includescommitted Kafka-compatible throughput that Kinesis would duplicate rather than use. Google CloudPub/Sub was set aside because standing up a second cloud provider's identity and networking stack forone service is not something a two-person platform team is resourced to maintain. NATS JetStream wasset aside on familiarity: Thornwell already runs RabbitMQ for two other internal queues and has neverrun NATS, so RabbitMQ carries the "message broker the team already knows" argument and NATS does not.That left four alternatives, including the baseline.
| ID | Alternative | Description | Do-nothing or do-minimum baseline? ||---|---|---|---|| A0 | Keep the current nightly batch upload | No new ingestion system. Each truck's onboard GPS log is extracted once it docks overnight and the batch file is uploaded into the existing data-warehouse job, as today. | Yes || A1 | Self-managed Apache Kafka | A Kafka cluster the platform team builds, configures and operates itself on the company's own compute, with replication factor and acknowledgment settings set per topic by the team. | No || A2 | Amazon MSK (provisioned) | A managed Kafka control plane: AWS creates, updates and deletes the cluster's brokers and replaces a failed broker automatically; the team still owns topic-level configuration. | No || A3 | RabbitMQ Streams, self-hosted | A RabbitMQ stream, run by the platform team on the same infrastructure that already hosts Thornwell's two other internal RabbitMQ queues. | No |
## Scoring Matrix
The do-nothing baseline is scored on every criterion below for comparison, but its weighted total isnot computed: it fails mandatory criterion C1, and a failed mandatory criterion drops an alternativefrom the ranking rather than merely cost it points.
| Criterion | Weight | A0: Nightly batch | A1: Self-managed Kafka | A2: Amazon MSK | A3: RabbitMQ Streams ||---|---|---|---|---|---|| C1: Replicates before ack | 15 | 0 (fails: a truck's positions sit only on that truck's own device until the nightly job runs, so nothing is replicated before the batch exists at all) | 8 (Kafka's replication factor is set per topic, and a committed message survives as long as one in-sync replica stays up, once the team sets `acks` and `min.insync.replicas` to require that) | 6 (AWS replaces a failed broker automatically, but this research found no Amazon MSK-specific statement of a per-topic replication or commit guarantee independent of Kafka's own documentation, so the score rests on broker-level failure handling rather than a message-level guarantee the team has independently confirmed for MSK) | 7 (RabbitMQ streams are persistent and replicated by default, and a publisher confirms are issued once the write has reached a quorum of stream replicas; the gap is that a replica does not explicitly flush to disk, so an uncontrolled shutdown can still lose data on that one node) || C2: Operational burden | 35 | 10 (nothing new to run) | 3 (the team owns cluster operations end to end with no prior production experience running one) | 7 (AWS owns the control-plane operations; the team still owns topic configuration and client-side settings) | 4 (self-hosted, but the team already operates two other RabbitMQ queues, so the operational pattern, if not the stream feature specifically, is familiar) || C3: Integration effort | 25 | 10 (nothing changes) | 5 (new client libraries, new topic schema, and a new consumer to feed the dashboard) | 6 (same client-side work as self-managed Kafka, with cluster provisioning handled by AWS instead of the team) | 8 (the team's existing RabbitMQ client code and monitoring carry over; only the stream-specific consumer logic is new) || C4: Cost (illustrative team estimate) | 15 | 10 (already paid for; no new spend) | 6 (compute cost for a cluster the team must size itself) | 5 (managed-service premium on top of the same compute) | 7 (runs on infrastructure already provisioned for the existing queues) || C5: Latency | 10 | 1 (next business day, by design) | 8 (sub-minute, estimated from the team's own prototype) | 8 (sub-minute, estimated from the team's own prototype) | 7 (sub-minute, estimated from the team's own prototype, slightly behind the other two because of the quorum-confirm round trip) || **Weighted total** | | Dropped (fails mandatory C1) | 52.0 | 64.0 | 62.0 |
## Sensitivity Analysis
* Weight or score tested: the weight on operational burden (C2), the number Marta and Dev argued over most, because neither of them has operated a message broker in production and they disagreed about how much that inexperience should count against both live broker options equally.* How far it would have to move before the recommendation changes: shifted from 35 down toward 30 of the 100 total points, moving the difference onto integration effort (C3). At a weight of 31, Amazon MSK and RabbitMQ Streams are tied at 63.6. At 30, RabbitMQ Streams leads, 64.0 to 63.5.* Does the recommendation survive this test: yes, but only just. Amazon MSK leads at any weight on operational burden above 31; at 31 the two tie, and at 30 RabbitMQ Streams takes the lead. A four-point move in a single weight erases the lead, and five points reverse it.
## Risk and Benefit
| Alternative | Risk | Benefit | Mitigation (if this becomes the recommendation) ||---|---|---|---|| Self-managed Kafka | If the team sets `acks` or `min.insync.replicas` wrong while learning the system, the committed-message guarantee is weaker than the score above assumes, and neither engineer has configured these settings before. | Finer control: replication factor and the acknowledgment setting are both set per topic, rather than inherited from a managed service's defaults. | Bring in a contractor with production Kafka experience for the first topic configuration, or send one engineer to training before go-live. || Amazon MSK | The broker-level auto-replacement this score credits is AWS's; this research did not independently confirm a per-topic replication or commit guarantee for MSK beyond what self-managed Kafka's own documentation states, so part of what the team is relying on for durability is inherited rather than separately verified for the managed service. | AWS owns cluster creation, update and deletion, and automatically mitigates or replaces an unhealthy broker, which is exactly the operational work the two-person rotation is least equipped to absorb. | Before committing operationally, have the team confirm the actual per-topic replication factor and acknowledgment settings it will run on MSK, rather than assuming they match self-managed Kafka's documentation by default. || RabbitMQ Streams | Without an explicit retention policy, a stream's log grows until the disk runs out, and a replica does not flush to disk automatically, so an uncontrolled shutdown can lose data for replicas on that one node. A stream also needs a quorum of its declared nodes available to function, so losing too many nodes makes it unavailable rather than merely degraded. | The team already runs two other RabbitMQ queues on the same infrastructure, so monitoring, alerting and the on-call runbook pattern already exist and do not need to be built from nothing. | Set an explicit retention policy before this carries fleet telemetry, and document the quorum-loss failure mode in the on-call runbook before go-live. |
## Recommendation
* Recommended alternative, or alternatives if genuinely close: Amazon MSK, narrowly ahead of RabbitMQ Streams, 64.0 to 62.0 of 100. Call this a near-tie rather than a clear win: the Sensitivity Analysis above shows a four-point move in one weight erases the lead. Self-managed Kafka is not competitive at any weight tested in that analysis. The nightly-batch baseline is excluded from the ranking entirely because it fails mandatory criterion C1.* Rows this depends on: C2 (operational burden), where MSK scores well because AWS owns the control-plane work the two-person rotation is least equipped to absorb, and C1, where the evidence for MSK's own durability guarantee is thinner than the evidence for self-managed Kafka's, a gap the Risk and Benefit section names directly.* Conditions under which this holds: stands only as long as operational burden's weight stays above 31 of the 100 points. This should be revisited if the two-person rotation gains a third engineer, if the team's RabbitMQ experience ends up outweighing its Kafka experience, or if the per-topic replication settings confirmed for MSK (see Risk and Benefit) turn out weaker than self-managed Kafka's.* Decision taken, and where it is recorded for good: adopt Amazon MSK for GPS telemetry ingestion, conditional on confirming the per-topic replication and acknowledgment settings before go-live. Recorded in Thornwell Freight's internal decision log as entry TF-DEC-2026-07.
## Dissent or Independent Review
* Who disagreed, and how it was addressed: Dev Marsh, the platform engineer who would carry most of the day-to-day operation of whichever broker is chosen, preferred RabbitMQ Streams, on the strength of the team's existing RabbitMQ experience, and argued the operational-burden weight of 35 overstated how much that experience gap actually matters once either system is running in steady state. Marta reworked Sensitivity Analysis using Dev's preferred weight of 28; at that number RabbitMQ Streams leads, 64.8 to 63.3. Dev accepted the result once the Sensitivity Analysis made the threshold explicit, rather than continuing to argue the number in the abstract, but flagged that the decision should be revisited after the first quarter of running MSK if the operational burden turns out closer to his estimate than to the team's.* Who checked the scoring, and how independent of the outcome they are: Sofia Reyes reviewed the completed matrix before this document was finalized and found no arithmetic error. She is not independent of the outcome: she owns the on-call rotation that will run whichever system is chosen, so she has a stake in the answer. No reviewer outside the platform and operations teams checked the scoring.
## References
* Apache Kafka documentation, replication and acknowledgment sections, read 2026-09-29, source for the replication-factor, `acks` and `min.insync.replicas` facts under self-managed Kafka in Criteria and Weights and the Scoring Matrix.* Amazon MSK documentation, cluster operations and broker-failure handling pages, read 2026-09-29, source for the control-plane and broker-replacement facts under Amazon MSK in the Scoring Matrix and Risk and Benefit.* RabbitMQ Streams documentation, persistence and replication pages, consulted 2026-09-30, source for the quorum-confirm, retention and disk-flush facts under RabbitMQ Streams in the Scoring Matrix and Risk and Benefit.* Thornwell Freight platform team, ingestion prototype notes, dated 2026-10-05, source for the sub-minute latency estimates in the Scoring Matrix's C5 row.* Thornwell Freight on-call rotation roster, current as of 2026-10-01, source for the two-person staffing fact behind the weight on operational burden in Criteria and Weights.* The new customer contract's service-level addendum, signed 2026-09-18, source for the "within the hour" alert requirement behind the weight on latency in Criteria and Weights.* Thornwell Freight platform team, internal cost estimate worksheet, dated 2026-10-02, source for the illustrative cost scores in the Scoring Matrix; no vendor price appears anywhere in this document.Provenance
Section titled “Provenance”The reasoning, the history and every source, in the repository:
- Companion - the long-form argument: why these sections, where the sources disagree, and what the bundle refuses to claim
- History - what changed in this bundle, and when
- Research log - every source consulted, with what each one actually supports
- Catalog metadata - the machine-readable record this page is generated from
Catalog record: 10 sections across 1 format(s).