Skip to content

Runbook

A step-by-step operational procedure for a recurring task or incident, written to be followed correctly under pressure.

A runbook is an operational document written to be executed, not just read. Its primary design constraint is that an operator who did not build the system must be able to complete the procedure correctly by following the steps sequentially, without needing to understand why each step exists. The format earns its place precisely because systems fail at unpredictable moments, and the operator handling the failure may be under time pressure, may have been paged at 3 AM, or may be unfamiliar with this specific system. A runbook makes the procedure survivable under those conditions.

The canonical runbook structure groups steps into phases that match the operator’s mental model of an operation: Prerequisites (what must be true before starting), Procedure (numbered steps with expected outputs), Verification (how to confirm the procedure succeeded), and Rollback (how to undo if something went wrong). Steps are numbered, not bulleted, because order matters. Each step should be atomic - one action, one expected output - so the operator knows at each point whether to continue or stop and escalate.

# [Runbook Title]
## Overview
[One sentence: what this procedure does and when to trigger it.]
## Prerequisites
- [ ] [Condition that must be true before starting]
- [ ] [Access, credentials, or tools required]
## Procedure
1. **[Action verb] [target]**
Expected output: [what you should see if this step succeeded]
2. **[Action verb] [target]**
Expected output: [what you should see if this step succeeded]
## Verification
[How to confirm the full procedure completed successfully.]
## Rollback
[Steps to undo this procedure if something went wrong, or "Not applicable" with justification.]
## Escalation
[Who to contact and how if this runbook does not resolve the situation.]

Recurring operational procedures where step order matters and a missed step causes failures, incident response playbooks that must be usable by engineers who did not build the system, deployment or release procedures that carry rollback risk, onboarding steps for new operators who need to run systems without full context, any procedure where a person unfamiliar with the system must complete it correctly the first time.

Conceptual or architectural explanations where the goal is understanding rather than execution, reference material that is looked up non-sequentially rather than followed step by step, one-off procedures that will never be repeated and do not warrant documentation overhead.

operator, direct-communicator, instructional, matter-of-fact, procedural

readme: A README orients a reader to a project - what it is, how to install it, and where to start. A runbook is not an orientation document; it is a procedure to follow in sequence during a specific operation. A README can be read in any order; a runbook must be followed step by step to a defined end state.

technical-reference: A technical reference lists commands, parameters, APIs, or configuration options for lookup. A runbook sequences those primitives into a specific operation with a defined start condition, end state, and rollback path. A reader consults a reference to find information; a reader follows a runbook to complete a task under operational conditions.

  • Numbered steps in strict execution order, one action per step
  • An expected output note after each step so the operator knows whether to continue or stop
  • A Prerequisites section listing conditions, access, and tools that must be true before starting
  • A Rollback section that either states how to undo the procedure or explicitly notes rollback is not applicable
  • Imperative verbs at the start of each step (“Run”, “Navigate”, “Verify”, “Copy”, “Confirm”)
  • A Verification section separate from the Procedure that confirms the full operation succeeded
  • Writing steps at the level of concepts rather than concrete commands or interface actions - An operator under pressure cannot fill in implicit steps; every command, click, and navigation path must be spelled out so the procedure works for someone who has never run it before.
  • Omitting the Rollback section because the author assumes the procedure is low-risk - Risk is assessed under normal conditions; a runbook is used under abnormal ones. Even “No rollback available” is a valuable signal that tells the operator to proceed with extra care.
  • Writing the runbook as a prose explanation of how the system works instead of a sequence of actions - That produces a readme or technical-reference, not a runbook; the reader needs to do something right now, not understand the system architecture.
  • Using a single numbered list for a procedure that has conditional branches (“If X, skip to step 7”) - Branching logic inside a numbered list is impossible to follow under pressure; conditional paths belong in separate sub-procedures or clearly labeled sections with their own step sequences.
  • Over-atomizes - breaks every action into sub-sub-steps until the procedure loses its navigational shape and an experienced operator cannot see the sequence for the individual steps - Group logically inseparable actions into single steps. The target reader is competent but context-free, not inexperienced. If an operator with basic system familiarity would never mentally separate two actions, they belong in one step.
  • Buries the operator in inline verification notes and conditional branches within the Procedure until the document requires more reading than acting, defeating the execute-under-pressure purpose - Consolidate post-step verification into the single Verification section. Reserve inline expected-output notes for steps where a silent failure would lead the operator to continue in a broken state.
Write as a Runbook. Use the canonical sections: Overview, Prerequisites, Procedure, Verification,
Rollback, and Escalation. In Overview: one sentence stating what this procedure does and when to
trigger it. In Prerequisites: a checklist of conditions, access, and tools that must be true
before starting. In Procedure: numbered steps with one action per step. End each step with an
expected output so the operator knows whether to continue or stop and escalate. In Verification:
state how to confirm the full operation succeeded. In Rollback: state how to undo the procedure,
or explicitly say rollback is not applicable and why. Start every step with an imperative verb
("Run", "Navigate", "Copy", "Verify"). Assume the reader is competent but has never run this
procedure before and may be under time pressure. Do not explain why steps work - write what to do.

See the Runbook template.

Operator, Direct Communicator, Instructional, Matter of Fact, Procedural

Columnist, Confessional, Playful

README, Technical Reference

On-Call Triage: Daily Async Standup (#team-standup)

Section titled “On-Call Triage: Daily Async Standup (#team-standup)”

Scan #team-standup for the day’s posts, confirm every @mention has a substantive response, and escalate any unresolved blockers - run this once daily between 10am and 11am Pacific on any day you are the designated on-call engineer.

  • You are listed as on-call for this week (confirm in #on-call-schedule)
  • You have access to #team-standup in Slack
  • You have 20-30 minutes available between 10am and 11am Pacific today
  • You have checked the team calendar for any PTO that reduces today’s expected post count
  1. Open #team-standup and filter to messages posted since yesterday’s triage session. Expected output: A set of posts from team members. The full team is 11 engineers across US Pacific, US Eastern, UK, and India timezones. Subtract anyone confirmed on PTO to get today’s expected headcount.

  2. Tally posts against today’s expected headcount. Expected output: A count. If fewer than 80 percent of expected posts have arrived and it is past 10:30am Pacific, continue to step 3. Otherwise skip to step 4.

  3. Send a direct message to each engineer who has not posted. Use this text: “Hey, no standup post yet today - any blockers?” Expected output: A reply or a channel post within the hour. If no reply after 60 minutes and the engineer is not on PTO, go to Escalation.

  4. Read every post in full. Identify every @mention that appears inside a “Blocked or at risk” field. Expected output: A list (possibly empty) of @mentioned names tied to a specific blocker. Posts with no @mention in that field require no immediate action beyond step 6.

  5. Reply in thread to each @mention. For each blocked item, respond with one of: a resolution, a diagnostic question, or a handoff naming the person who can unblock them. Expected output: Every @mention has a thread reply from you. Target completion within 30 minutes of your first read of the channel.

  6. Scan for “at risk” items with no @mention. Identify any at-risk line where the engineer did not tag a resolver. Expected output: A list (possibly empty) of at-risk items with no owner named.

  7. Reply in thread on each unowned at-risk item. Ask whether the engineer needs help and whether you should loop in a specific person. Expected output: A thread is open on every unowned at-risk item. You do not need to resolve it yourself at this step.

  8. Post a triage summary in #on-call-log. Format: “Standup triage [date]: [N] posts received, [N] blockers found, [N] resolved, [N] escalated.” Expected output: A message visible to the engineering manager confirming triage is complete for the day.

Every @mention in today’s #team-standup posts has a thread reply from you. Every at-risk item with no @mention has a thread reply asking whether help is needed. Your summary message is posted in #on-call-log.

Not applicable. Triage is a read-and-respond procedure with no reversible system state. If you replied to the wrong thread or gave incorrect guidance, post a correction reply in the same thread immediately.

  • Blocker you cannot resolve: Tag the engineering manager in the thread with one sentence describing the blocker and why you are escalating.
  • Engineer unreachable after 60 minutes and not on PTO: Send a direct message to the engineering manager with the engineer’s name and last known context.
  • At-risk item that could affect the Thursday working session agenda: Add it to docs/thursday-agenda.md and notify the engineering manager.
  • Process questions not covered here: Check docs/playbook.md first. If still unresolved, ask in #on-call-schedule.