Design Document
An engineering document that explores a design space and justifies an approach before implementation.
Design Document
Section titled “Design Document”A design document works through how a non-trivial piece of software will be built, before it is built: the problem, the constraints, the proposed design, the alternatives weighed, and the risks. Where an ADR records a single decision already made, a design doc is the working space where the design is reasoned through before decisions solidify. Its audience is the engineers who will build and review the system - it exists to surface problems early, coordinate understanding, and create a record of the reasoning behind the implementation.
The hallmarks of a good design doc are proportionality and specificity. It goes deep enough on schemas, APIs, component boundaries, and data flows to give reviewers something real to push back on - but it stays focused on the problem being solved rather than general principles. A design doc that covers everything equally covers nothing usefully.
Design docs live in the space between “we are going to build X” (a PRD) and “we decided to do it this way” (an ADR). They are working documents: they may be updated as understanding evolves, and they expire when the system is built and the decisions have been absorbed into ADRs and operational docs. Their value is in the conversation they make possible before a line of production code is written.
Canonical template
Section titled “Canonical template”# [System / Feature Name] Design Document
## Status[Draft | In Review | Accepted | Superseded]
## Problem[What problem is being solved. The constraints that shape the solution space.]
## Proposed Design[How the system will work. Schemas, APIs, component boundaries, data flows.Be specific enough that a reviewer can evaluate the approach.]
## Alternatives Considered[Other approaches evaluated and why they were not chosen.]
## Risks and Open Questions[What could go wrong. What is still uncertain.]
## Appendix (optional)[Supporting diagrams, data models, or reference material.]When to use
Section titled “When to use”Planning a non-trivial feature or system before implementation begins, coordinating design across multiple engineers or teams, surfacing risks and open questions before a line of code is committed, creating a reviewable artifact that lets engineers evaluate the implementation at the concrete level - schemas, APIs, component boundaries - before code is written, documenting the reasoning behind an implementation for future maintainers.
When not to use
Section titled “When not to use”Small, self-contained changes where the implementation approach is obvious to any reviewer, after the system is already built (write ADRs for the decisions instead), when a decision about direction is still open and needs broad input before committing to build (write an RFC instead).
Pairs well with
Section titled “Pairs well with”pragmatic-architect, senior-consultant, candid, matter-of-fact, problem-solution, comparison-contrast
Often confused with
Section titled “Often confused with”adr: An ADR records a single decision already made and its consequences. A design doc is the working space that precedes those decisions - one design doc typically generates several ADRs as the design is reviewed and accepted.
prd: A PRD defines what should be built and why - its audience is product stakeholders. A design doc specifies how the engineering solution will be built - schemas, APIs, component boundaries, data flows - and its audience is the engineers who will implement and review the work.
rfc: A design doc is the implementation specification for something already being built; its currency is technical specificity - schemas, APIs, component boundaries, data flows. An RFC is a comment-seeking proposal for a design direction that invites pushback before a decision. A design doc assumes the decision to build has been made and works out how; an RFC is still asking whether and how to proceed.
- A Problem or Background section that names the constraints shaping the solution space
- A Proposed Design section committing to concrete implementation specifics - schemas, APIs, component boundaries, or data flows - that an RFC would not yet pin down
- An Alternatives Considered section that shows what was evaluated and why it was rejected
- A Risks and Open Questions section that names what could go wrong or is still uncertain
- A Status header (Draft, In Review, Accepted, Superseded) tracking where the document stands
- Technical specificity at the implementation level, not the product requirement level
- Audience is the engineers who will build or review the system, not product stakeholders
Anti-patterns
Section titled “Anti-patterns”- Writing at the product requirement level - what to build and why - instead of the implementation level - how to build it - A design doc that describes desired outcomes without specifying schemas, interfaces, or component boundaries is a PRD, not a design doc; engineers cannot evaluate or build from it.
- Omitting Alternatives Considered or listing alternatives without saying why they were rejected - The reasoning behind the chosen approach is where design docs earn their value; a list of alternatives without rejection rationale reduces to a menu, not a design.
- Scoping the document to the entire system rather than the specific change or addition being designed - System-wide scope turns a design doc into unmaintainable reference documentation; each design doc should cover one bounded change or addition.
- Soliciting a vote on whether to proceed rather than inviting technical review of how to proceed - That is the job of an RFC; a design doc assumes the decision to build has been made and works through the implementation, not the direction.
Failure modes
Section titled “Failure modes”- Over-specifies too early - locks in implementation details before the team has validated the approach, so review becomes defensive rather than generative - Flag uncertain areas explicitly in Risks and Open Questions; keep early drafts focused on structure and interfaces before committing to implementation minutiae.
- Expands into a treatise - grows to cover every edge case and becomes unreadable before the first reviewer reaches the Proposed Design section - Aim for the minimum specificity needed for a reviewer to evaluate the approach; move exhaustive detail to appendices or linked documents.
Instruction
Section titled “Instruction”Write as a Design Document. Work through how the system will be built, not what it shoulddo. Use these sections: Problem (the constraints shaping the solution space), ProposedDesign (concrete specifics: schemas, APIs, component boundaries, data flows - enough fora reviewer to push back on the approach), Alternatives Considered (other approaches andwhy they were rejected), Risks and Open Questions (what could go wrong and what is stilluncertain). Be technically specific. Your audience is engineers who will build and reviewthe system. Do not write at the product requirement level - that is a PRD. Do notsolicit a decision on whether to proceed - that is an RFC. Write at the implementationlevel, assuming the decision to build has been made.Template
Section titled “Template”See the Design Document template.
Related
Section titled “Related”Pairs well with
Section titled “Pairs well with”Pragmatic Architect, Senior Consultant, Candid, Matter of Fact, Problem-Solution, Comparison-Contrast
Avoid with
Section titled “Avoid with”Playful, Pastoral, Urgent, Reverent
Often confused with
Section titled “Often confused with”Architecture Decision Record, Product Requirements Document, RFC (Request for Comments)
Examples
Section titled “Examples”- Whether the team should move to async-first standups
- Designing a sustainable morning routine
- Choosing Postgres vs DynamoDB for a new service
- Telling stakeholders a committed feature is being cut this quarter
- Getting a new engineer productive in their first two weeks
- Writing to thank a mentor who shaped your career
- Reflecting on keeping a discipline of rest
- Marking a long-serving colleague's departure
- Marking the team shipping a hard, long project
- Arguing a public position on return-to-office
- Announcing a new product to an outside audience
- A personal year-end reckoning with a difficult year
Async Standup System Design Document
Section titled “Async Standup System Design Document”Status
Section titled “Status”Accepted
Problem
Section titled “Problem”The engineering team (11 engineers, 4 timezones) runs a synchronous daily standup at 9am Pacific. This format fails on three fronts.
Timezone asymmetry. 9am Pacific is 9:30pm IST. The three India-based engineers averaged 3.2 out of 5 standup appearances per week in Q1, against 4.6 for US-based engineers. The gap is not a performance issue; the schedule is structurally unfair.
No persistence. Status exchanged verbally disappears when the call ends. We documented three Q1 incidents where an engineer spent over an hour on a problem already surfaced and resolved in a previous standup. The meeting produces no searchable record.
Low signal-to-noise ratio. The standup averages 14 minutes. Roughly 4 minutes drives an action - a blocker raised, a dependency flagged, a context shared. The remaining 10 minutes is status nobody needed to respond to.
Constraints shaping the solution:
- Cannot require attendance outside normal working hours for any engineer
- Must preserve the coordination value of the standup: blocker detection and dependency surfacing
- Must produce a persistent, searchable record
- Tooling must stay within Slack; no new paid tools unless value clearly exceeds cost and complexity
Proposed Design
Section titled “Proposed Design”Post schema
Section titled “Post schema”Each engineer posts one message to #team-standup by 10am their local time. The message follows a three-field template:
Shipped: - <what completed since last post>
In progress: - <current focus>
Blocked or at risk: - <what is stuck and who or what resolves it, @mentioning the responsible party>Empty fields carry the literal value “nothing today.” Fields are not optional - an absent field is ambiguous between “I forgot” and “nothing to report.” The 10am local cutoff creates a clear posting window that gives all four timezones time to post before the Pacific-based on-call scan runs.
Channel architecture
Section titled “Channel architecture”One channel: #team-standup. A pinned message carries the template and two exemplar posts (one from a high-load day, one from a quiet day). No bots. No automated summaries. The channel is a log, not an inbox.
The /standup Slack message shortcut pre-fills the template into the compose box. It is optional convenience, not the mechanism - typing the three headers manually is equally valid.
On-call triage role
Section titled “On-call triage role”The on-call engineer scans #team-standup once, between 10am and 11am Pacific. Their responsibility is narrow:
- Confirm every engineer has posted; DM late posters
- Ensure every
@mentionin a Blocked field has received a substantive reply within 30 minutes of the scan
The on-call is not a summarizer. They do not re-broadcast channel contents. Thread replies are the response surface; a new top-level message is only appropriate for a team-wide blocker affecting multiple people.
This role replaces the meeting facilitation slot already on the on-call rotation.
Thursday working session
Section titled “Thursday working session”The 60-minute weekly slot at 8am Pacific / 8:30pm IST replaces the five daily sync standup slots. Agenda lives in docs/thursday-agenda.md, maintained by the engineering manager. If the doc has no items by Wednesday 5pm Pacific, the session is cancelled for that week.
This is not a standup in async clothing. It is reserved for discussion that requires real-time exchange: design review, cross-team dependency negotiation, or issues that carry nuance the three-field template cannot hold. Status reporting is not a valid agenda item.
Data flow
Section titled “Data flow”Engineer post (by 10am local) --> #team-standup (persistent, searchable) --> On-call scan (10am-11am Pacific) --> @mention reply (SLA: 30 min during business hours) --> Thursday agenda item (if discussion needed but not urgent)Alternatives Considered
Section titled “Alternatives Considered”Geekbot or equivalent async standup bot. Rejected. Geekbot solves the scheduling problem and adds structure, but it introduces a paid tool, a data store outside Slack’s native search, and vendor dependency for a process the Slack-native approach handles. Cost and complexity are not justified given functional parity.
Rotating the meeting time. Rejected. A rotating time makes the schedule equitable on average but does not create information persistence. It also adds coordination overhead and removes engineers’ ability to hold a stable recurring block for deep work, since the standup slot is always moving. Solves one of three problems; the other two remain.
Eliminating the standup entirely. Rejected. The coordination value - blocker surfacing, dependency flagging, context sharing - is real, even if the signal is buried in noise under the current format. Removing the standup removes the mechanism without replacing the value.
Moving standup to 12pm Pacific. Rejected. 12pm Pacific is 12:30am IST the next day, shifting the timezone burden from late evening to late night for India. Worse, not better.
Risks and Open Questions
Section titled “Risks and Open Questions”Post quality drift. If engineers write long-form updates instead of three-bullet summaries, the channel becomes slower to scan and on-call triage load increases. Mitigation: pin two exemplar posts from day one; set an explicit expectation in the process playbook that each field should be readable in under 15 seconds.
On-call triage load. The target is 10 minutes per morning for the channel scan. If blockers surface earlier and louder than they did in the sync model, actual time could run higher. If the role consistently exceeds 20 minutes, consider splitting triage responsibility by timezone cluster.
Participation cliff. The format’s coordination value scales with participation. If one or two engineers consistently miss the cutoff, the channel becomes unreliable as a team signal. Mitigation: on-call DMs late posters; the 30-day trial retro surfaces participation data to catch this early.
Social cohesion. The sync standup provided a daily shared moment that the async format does not replicate. The Thursday session is a partial substitute. This is a real trade-off accepted because the equity and persistence gains are larger, but qualitative feedback in the trial retro should track it explicitly.
Open question: cutoff enforcement. The 10am local cutoff is a norm, not a hard technical constraint. Whether to add a bot reminder at 9:45am local is deferred until after the 30-day trial produces data on late-posting patterns.
Open question: holiday handling. Behavior on local public holidays (UK bank holidays, Indian national holidays, US federal holidays) is not yet specified. Deferred to trial retro.
Morning Routine v3.0 - Design Document
Section titled “Morning Routine v3.0 - Design Document”Status
Section titled “Status”In Review
Problem
Section titled “Problem”Two prior attempts at an intentional morning routine (v1.0 and v2.0) failed within a week each. Both failures trace to the same root cause: I removed a behavior (phone scrolling) without substituting a replacement sequence. The vacuum collapsed within days. This document specifies the implementation for v3.0, designed to be specific enough that my accountability partner can evaluate it before I start and that I cannot quietly renegotiate terms once I am tired.
Constraints shaping the solution space:
- Time window: 6:30am to 7:30am on weekdays. Family obligations begin at 7:30am (breakfast, school dropoff). This is the only reliably discretionary hour.
- Device dependency: The phone is the primary failure mechanism. Any design that leaves the phone in the bedroom will reproduce the v1.0 and v2.0 failure mode: I will reach for it before my feet hit the floor.
- Sequence rigidity: I cannot make good decisions before 7am. A fixed ordered sequence, not a menu of options, is required. Decision fatigue at 6:31am is a known failure mode in prior versions.
- Complexity ceiling: Prior attempts added modules until the morning felt like a second job. The total design must fit inside 60 minutes with buffer remaining for the unplannable parts of a family morning.
Proposed Design
Section titled “Proposed Design”The routine is a four-module pipeline running in fixed order. Modules do not branch and do not unlock each other; if a module is skipped, the remaining modules still run from where the schedule permits. The sequence is: Water, Light, Movement, Planning.
Module 1: Water
Section titled “Module 1: Water”Duration: 5 minutes (absorbed into waking transition)
Specification: 500ml water consumed within 5 minutes of waking. A glass is pre-staged on the nightstand the night before as part of the shutdown ritual (see Risks). Coffee does not substitute and follows later.
Role in pipeline: Functions as a contextual trigger. Picking up the glass signals to me that the protocol is running. This is the cheapest module and also the one that sets the others in motion.
Module 2: Light
Section titled “Module 2: Light”Duration: 10 minutes
Specification: Outdoors preferred (back door, front step). Fallback when outdoor exposure is genuinely blocked by weather: stand at the largest available window with curtains fully open. Headphones permitted. Screens do not substitute as a light source.
Role in pipeline: Early light exposure is the input that anchors the rest of the sequence. The ordering hypothesis (light before movement) is the implementation decision most worth validating empirically; see Open Questions.
Module 3: Movement
Section titled “Module 3: Movement”Duration: 15 minutes
Specification: Heart rate elevated, not crushed. Acceptable forms: 15-minute walk (default), bodyweight circuit defined in protocol/movement.md, or a stretch-only session when recovery is a factor. A gym session does not satisfy this slot - it exceeds the time budget and front-loads decision overhead. The bar is “body is moving,” not “workout complete.”
Role in pipeline: Runs after light to use the early-morning cortisol window without competing with Module 2. Running it before light inverts the sequencing rationale.
Module 4: Planning
Section titled “Module 4: Planning”Duration: 10 minutes
Specification: Paper and pen only. Output: three items I intend to close today, written in a dated notebook entry. The phone is not present for this module. A log row for log/days.csv is filled in on the same page at the end of this step and transcribed to the file at week’s end.
Role in pipeline: Planning is the only cognitively demanding module. Placing it last means 25-30 minutes of lower-demand activity have reduced sleep inertia before I try to set priorities for the day.
Enforcement mechanism: phone-in-kitchen rule
Section titled “Enforcement mechanism: phone-in-kitchen rule”The phone is plugged in on the kitchen counter, face down, before going to bed each weeknight. It is not retrieved until Module 4 is complete. The phone is not brought to the bedroom.
This is the load-bearing constraint for the entire protocol. Without a hard location rule, the phone becomes accessible at the moment of lowest willpower (waking). Prior versions relied on willpower; this version relies on physical separation.
Time budget
Section titled “Time budget”| Module | Duration | Cumulative |
|---|---|---|
| Water | 5 min | 0:05 |
| Light | 10 min | 0:15 |
| Movement | 15 min | 0:30 |
| Planning | 10 min | 0:40 |
| Buffer | 20 min | 1:00 |
The 20-minute buffer covers getting dressed, making coffee, and whatever the morning actually contains. It is not a module. Do not schedule it.
Logging schema
Section titled “Logging schema”Each weekday morning appended to log/days.csv:
date, wake_time, completed (yes/no/partial), steps_skipped, one_word_mood, notesCompletion criterion: all four modules run, in order, on the same morning. “Partial” is not a passing grade for the 30-day success metric. It logs what happened without inflating the completion count.
Alternatives Considered
Section titled “Alternatives Considered”Earlier wake time (5:30am instead of 6:30am). Rejected. Fixing the content of the existing hour before adding a new one is the correct sequencing. Adding 60 minutes while the current hour is broken is premature optimization. Revisit only after a successful 30-day run at 6:30am.
Habit-tracking app. Rejected. Adds a screen to a protocol whose primary leverage is screen removal. The paper log is slower and less convenient; that is intentional. Moving to a digital log would reintroduce the phone into the planning step.
Variable module menu (pick any three in any order). Rejected. Prior v2.0 design allowed flexible ordering; Module 3 (movement) was consistently skipped. Choice at 6:31am is the failure mode, not a feature.
Meditation substituted for planning. Considered as a Module 4 variant. Rejected for v3.0 because the planning output (three items, written down) is directly testable. If I cannot measure completion, I cannot review the protocol at 30 days. Meditation may be added as an optional extension after the first 30-day review.
90-day commitment frame. Rejected based on direct prior evidence: tried at this duration and quit on day 11. A 30-day frame with a formal review point is the minimum viable commitment. Extend only after the first 30-day completion.
Risks and Open Questions
Section titled “Risks and Open Questions”Risk: shutdown ritual is an undocumented dependency. The phone-in-kitchen rule and the pre-staged water glass both require a consistent shutdown ritual the night before. If the shutdown ritual is skipped (late night, travel, family exception), the morning protocol launches without its setup. Mitigation: document the shutdown ritual before day 1 and treat it as part of the protocol. Current bedtime is approximately 11:30pm; the protocol likely requires compressing that to 10:30pm to make 6:30am viable without sleep debt.
Risk: family friction. My spouse’s morning coffee window runs 6:45 to 7:15am. Modules 2 and 3 overlap with that window and involve leaving and re-entering the house. I have not confirmed that this works for the household. This needs a direct conversation before launch. An assumption of tolerance is not a mitigation.
Risk: West Coast Slack messages missed. Colleagues on West Coast time send messages that arrive before 7:30am. Risk is accepted: urgent issues can reach me by phone call. The phone is in the kitchen and can receive calls even during the protocol.
Open question: weekends. The protocol is specified for weekdays. Family dynamics on weekends differ (no school dropoff, no fixed 9am start). Running the same protocol on weekends may be correct; a lighter variant may be better. Decision deferred to Week 2 based on what the first weekend actually reveals.
Open question: travel variant. Module 2 (light) and Module 3 (movement) degrade in hotel environments. A travel variant needs to be specified before the first travel day in Month 1, not improvised in a hotel room at 6:45am.
Open question: light-before-movement ordering hypothesis. The current sequencing (light before movement) is based on reasoning about early-morning cortisol timing. It has not been tested against movement-first. If Month 1 completion is high enough to control the variable, this is the ordering hypothesis most worth testing empirically in Month 2.
Appendix
Section titled “Appendix”Prior version summary
Section titled “Prior version summary”| Version | Period | Duration | Failure mode |
|---|---|---|---|
| v1.0 | 2025-10 | 4 days | Phone removal with no replacement; vacuum collapsed |
| v2.0 | 2025-12 | 6 days | Journaling added but no fixed sequence; movement skipped consistently |
Accountability partner handoff
Section titled “Accountability partner handoff”This document is the artifact shared with the accountability partner before day 1. Agreed escalation criterion: if 30-day completion falls below 70 percent of weekdays, schedule a session to diagnose before making any changes to the design.
Notification Service - Database Layer Design
Section titled “Notification Service - Database Layer Design”Status
Section titled “Status”In Review
Problem
Section titled “Problem”Lattice Notify is building a real-time notification delivery system. Before implementation begins, the team needs to commit to a database technology and a concrete design for how notification events are stored, queued, and delivered.
Access pattern. The write pattern is fanout: one source event generates one notification row per recipient and one delivery job per channel. Writes are insert-only. The dominant read pattern is user-scoped point-lookups: “last 50 notifications for user X.” Deletes are rare (30-day TTL via batch job). No analytical queries run against this table.
Scale envelope. At launch: 500K notification events per day, roughly 6 events per second sustained with peaks around 30/sec during business hours. A 10x growth scenario (5M events/day) depends on a pending partnership deal that has not closed.
Team constraint. 8 backend engineers, 4-person on-call rotation. The team has operated Postgres in production at this scale. The team has no production DynamoDB experience.
What this design must answer. Schema layout, write and read data flows, component boundaries between the notification service and upstream callers, worker pool architecture, and the operational surface the on-call rotation inherits.
Proposed Design
Section titled “Proposed Design”Technology: Postgres, new notifications schema on the existing primary cluster.
Schema
Section titled “Schema”CREATE TABLE notifications.events ( id UUID PRIMARY KEY DEFAULT gen_random_uuid(), workspace_id TEXT NOT NULL, user_id TEXT NOT NULL, channel TEXT NOT NULL, -- 'in_app' | 'email' | 'slack' template TEXT NOT NULL, payload JSONB NOT NULL DEFAULT '{}', delivered_at TIMESTAMPTZ, created_at TIMESTAMPTZ NOT NULL DEFAULT now());
CREATE INDEX ON notifications.events (user_id, created_at DESC);CREATE INDEX ON notifications.events (workspace_id, created_at DESC);
CREATE TABLE notifications.jobs ( id UUID PRIMARY KEY DEFAULT gen_random_uuid(), event_id UUID NOT NULL REFERENCES notifications.events(id), channel TEXT NOT NULL, status TEXT NOT NULL DEFAULT 'pending', -- 'pending' | 'processing' | 'done' | 'dead' attempts INT NOT NULL DEFAULT 0, next_attempt TIMESTAMPTZ NOT NULL DEFAULT now(), error TEXT, created_at TIMESTAMPTZ NOT NULL DEFAULT now());
CREATE INDEX ON notifications.jobs (status, next_attempt) WHERE status IN ('pending', 'processing');Write path
Section titled “Write path”- Caller writes one row to
notifications.eventsand one row per delivery channel tonotifications.jobsin a single transaction. - At commit, the caller issues
pg_notify('notification_ready', event_id::text). - The worker pool, listening on
notification_ready, wakes, claims the job viaUPDATE ... RETURNINGwith a short lock timeout, and executes delivery. - On success:
jobs.status = 'done',events.delivered_at = now(). After 3 failed attempts:jobs.status = 'dead'for dead-letter review.
Read path
Section titled “Read path”- User inbox reads go to a read replica:
SELECT * FROM notifications.events WHERE user_id = $1 ORDER BY created_at DESC LIMIT 50. No joins required for the inbox case. - Cross-schema joins (enriching notifications with account display names for digest email) run against the primary and are bounded to under 1 request per second.
Component boundaries
Section titled “Component boundaries”notification-serviceowns thenotificationsschema exclusively. No other service writes to it directly.- Upstream callers use
NotifyClient.send(user_id, channel, template, payload). The client opens a connection, writes the two rows, and callspg_notify. It has no knowledge of delivery mechanics. - The worker pool runs as a separate process in the same deployment unit. It is not called over a network boundary.
- Dead-letter rows (
status = 'dead') are drained by a scheduled job on a separate schedule from the worker pool; no automatic retry beyond 3 attempts.
Read replica provisioning
Section titled “Read replica provisioning”One read replica at launch. Promote to two if the inbox read rate exceeds 500 req/sec sustained. Jordan owns the dashboard metric that triggers this decision.
Alternatives Considered
Section titled “Alternatives Considered”DynamoDB with a single-table design
Section titled “DynamoDB with a single-table design”A composite key of (PK = user_id, SK = created_at#event_id) serves user inbox reads as a single Query with no secondary index. Fan-out writes land in parallel via batch-write. This is the access-pattern-optimal design.
Why not chosen. DynamoDB fits the access pattern well. The blocking constraint is operational, not architectural. The team has no production DynamoDB experience. Adding DynamoDB doubles the on-call surface: a second runbook, a second monitoring story, a second class of alerts. At 500K events/day the Postgres design is within the team’s demonstrated operating envelope, and the access-pattern advantage of DynamoDB does not outweigh the operational cost at this load. Marcus documented this tradeoff in experiments/notify-ddb/ after completing the spike. The 5M events/day revisit threshold re-opens DynamoDB consideration at the point where the access-pattern cost becomes dominant.
DynamoDB + SQS for the delivery queue
Section titled “DynamoDB + SQS for the delivery queue”Events land in DynamoDB; delivery jobs route through SQS, consumed by Lambda workers.
Why not chosen. This adds SQS and Lambda to the operational surface on top of DynamoDB. The team did not evaluate it in detail because DynamoDB was eliminated on operational grounds first. The surface area is strictly larger than the DynamoDB-only option.
Redis-backed job queue (Sidekiq or BullMQ pattern)
Section titled “Redis-backed job queue (Sidekiq or BullMQ pattern)”Events stay in Postgres; delivery jobs route through Redis rather than pg_notify.
Why not chosen. Adds a second datastore with no clear benefit over pg_notify at launch scale. pg_notify is adequate to roughly 2K messages per second; the design targets 30/sec peak. Revisit only if the team adopts Redis for another service-wide reason.
Risks and Open Questions
Section titled “Risks and Open Questions”1. Postgres partitioning at scale (owner: Ana)
At roughly 3M events/day sustained, the notifications.events table accumulates around 1 billion rows in a year. Range partitioning on created_at handles this, but the live migration is non-trivial. This is a known roadmap item targeted for Q4 if growth tracks projection. If the Slack partnership deal closes and 10x growth arrives in under 12 months, the team hits this window earlier. Mitigation: Priya monitors deal timing and commits to giving the team 30 days of warning before the 5M events/day threshold is breached.
2. pg_notify message loss on worker restart (owner: Sam)
pg_notify delivers only to active listeners. A worker restart while events are in flight drops those notifications. The worker pool must execute a startup poll - SELECT * FROM notifications.jobs WHERE status = 'pending' AND next_attempt <= now() - on startup and on a 30-second fallback schedule to recover missed events. Sam needs to confirm this fallback is in the implementation spec before the schema is finalized on 2026-05-20.
3. Dead-letter volume baseline (open question)
No measured delivery failure rate exists yet. Before first production traffic (target: 2026-05-29), the team needs an expected daily volume of status = 'dead' rows and an owner for reviewing them. Without this, the dead-letter drain job has no alerting threshold.
4. Worker pool connection count (owner: Jordan)
Each worker process holds one persistent LISTEN connection to the primary. The pool size at launch is not yet set. Jordan should add worker connection count to the on-call dashboard alongside queue depth and write rate by 2026-05-22, so the team enters launch with visibility into connection pressure.
Appendix
Section titled “Appendix”Event volume estimates
Section titled “Event volume estimates”| Scenario | Events/day | Avg events/sec | Peak events/sec |
|---|---|---|---|
| Launch | 500K | 6 | ~30 |
| Revisit threshold | 5M | 58 | ~290 |
References
Section titled “References”experiments/notify-ddb/- DynamoDB spike and access-pattern analysis (Marcus)- ADR-0023 (draft) - decision record; to be marked Accepted at the 2026-05-16 11am sync with Priya
#notify-arch- architecture discussion thread
Insights CSV Export - Design Document
Section titled “Insights CSV Export - Design Document”Status
Section titled “Status”Draft - September 12, 2026 Owner: Dario Reyes (engineering)
Problem
Section titled “Problem”ADR-0027 established that the Insights dashboard will not ship in Q3 because the billing-system migration consumed all available engineering capacity. The ADR commits to shipping a CSV export of the Insights data layer before September 30 as a stopgap, giving the four key accounts access to their data while the full dashboard is built in Q1 2027.
This document specifies how the CSV export will be implemented. The constraints that shape the solution space:
- Timeline. Backend endpoint must be complete by September 19; frontend integration and QA by September 24; release on September 26. September 30 is a hard deadline - there is no slip past end of quarter.
- Staffing. Dario Reyes is the sole engineering owner. One engineer, two weeks of capacity.
- Data availability. The underlying event data is already queryable from the existing analytics database. No new data collection, pipeline work, or schema additions are required.
- Self-serve requirement. Exports must complete without support involvement. Any flow requiring manual triggering by the team, a waiting period longer than a few seconds on typical accounts, or a follow-up communication to retrieve the file is out of scope for this timeline.
- Scope boundary. This is a stopgap, not a preview of the Q1 dashboard. It must not require infrastructure that would overlap with or create dependencies on the Q1 Insights build starting October 6.
Proposed Design
Section titled “Proposed Design”The export is a synchronous streaming HTTP endpoint attached to the existing data access layer.
Request flow
Section titled “Request flow”- The user navigates to Settings > Data and Analytics > Export in the Meridian web app.
- A “Download CSV” button issues a
GET /api/v1/insights/exportrequest with the user’s session cookie. - The server authenticates the request through the existing session middleware, resolves the account ID, and delegates to
ExportService. ExportServiceopens a cursor onEventQueryRepositoryscoped to the resolved account ID and begins streaming rows.- The HTTP response is returned with
Content-Type: text/csvandContent-Disposition: attachment; filename="insights-{account_id}-{export_date}.csv", writing rows in chunks as the cursor advances. - The stream closes when the cursor is exhausted. No temporary file is written to disk; no object storage is involved.
CSV schema
Section titled “CSV schema”Each row represents one event. Column order is fixed:
user_id UUID of the user who triggered the eventevent_name The recorded action (e.g., page_view, feature_used)event_timestamp UTC timestamp, ISO 8601 format (e.g., 2026-09-01T14:32:00Z)session_id UUID grouping events within a single sessionplan_tier The account plan active at the time of the eventThe header row is always emitted first. All string fields are quoted. Timestamps are always UTC with no localization. The export covers all events from account creation through the end of the previous calendar day (UTC).
Component boundaries
Section titled “Component boundaries”Three components are involved. Two are new; one is pre-existing and requires no modification.
ExportController(new): Thin HTTP handler. Validates the authenticated session, resolves the account ID from the session context, and delegates toExportService. Pipes the streaming response back to the client. Returns200on success; returns a structured error response if the account cannot be resolved or the repository query fails. Contains no business logic.ExportService(new): Owns the export logic. Opens a cursor onEventQueryRepository, iterates rows, serializes each to a CSV line, and writes to the response writer. Emits the header row before the first data row. Enforces the export scope: all events through end of previous calendar day, UTC.EventQueryRepository(existing): Already used by internal analytics tooling and the billing migration audit queries. Wraps the analytics database.ExportServicecallsqueryAllByAccount(accountId, endDate)on this interface. No modifications to the repository.
Authentication and authorization
Section titled “Authentication and authorization”The endpoint sits behind the existing session middleware that guards all /api/v1 routes. No new auth logic is introduced. Users can only export data for the account attached to their session. Cross-account access is not possible through this interface.
Frontend
Section titled “Frontend”The Settings page gains one new section: Data and Analytics. It contains a single “Download CSV” button. The button is disabled while a request is in flight and re-enabled on response. A browser-native download dialog handles file receipt on completion. Error states (export failure, session expiry) surface as inline text below the button. No other UI changes are in scope.
Alternatives Considered
Section titled “Alternatives Considered”Async generation with email delivery
Section titled “Async generation with email delivery”Generate the CSV in a background job and email the user a download link when complete. This is the standard pattern when generation can take minutes or when file sizes are large enough that a synchronous response would time out.
Rejected. The implementation requires a background job queue, transient file storage, and an outbound email trigger - none of which are plumbed for this endpoint today. Adding that infrastructure would consume time the team does not have before September 26 and would create a dependency on the email delivery stack for a feature that will be superseded in Q1. The synchronous streaming approach fits within the two-week window without new infrastructure.
Pre-generated daily exports in object storage
Section titled “Pre-generated daily exports in object storage”Generate each account’s export nightly and serve downloads from a stable per-account URL. Reduces per-request compute cost; downloads are always fast.
Rejected. Requires a nightly generation job, IAM scoping per account, and pre-signed URL generation on download. The object storage bucket structure would likely overlap with infrastructure the Q1 Insights build will want to own and design for its own purposes. Introducing it now as a stopgap creates unplanned technical debt for the Q1 team. The synchronous approach requires no shared infrastructure.
Paginated export across multiple smaller files
Section titled “Paginated export across multiple smaller files”Break the export into daily or monthly segments and let users download each independently.
Rejected for this scope. Most accounts will not produce file sizes that require chunking. Pagination adds UI complexity - which segments are available, which have been downloaded, how to merge them for analysis - that is out of proportion to the two-week timeline. If large-account export proves problematic after launch, pagination can be added as a patch before Q1 without altering the schema or the endpoint contract.
Risks and Open Questions
Section titled “Risks and Open Questions”Large account file sizes. Accounts with several years of event history may produce exports in the range of 50-100 MB. Synchronous streaming handles this without in-memory accumulation, but client-side download behavior at that file size should be validated during QA. If any of the four key accounts are known to have high event volume, test against representative data before September 24.
Billing release regressions. The billing-system migration goes to production the week of September 19. The CSV backend endpoint is due the same week. If production regressions surface and require Dario Reyes’s attention, frontend integration and QA may slip from September 24 toward September 30. September 30 is a hard deadline; there is one week of float, but it should be treated as a wall, not a buffer.
Rate limiting. No decision has been made on how frequently a user can trigger exports. For the Q3 release, no rate limiting is applied. If high-frequency export requests become a load concern after launch, a per-account daily limit can be added as a patch; no schema or API contract change is required to do so.
Q1 data contract. The column schema defined here represents the canonical shape of the Insights event data that the Q1 dashboard will visualize. The Q1 engineering design document (kickoff October 6) should treat these columns and the EventQueryRepository interface as established contracts and build on them rather than defining a parallel data model. Divergence here would require a migration before the dashboard ships.
Onboarding Bootstrap Tooling Design Document
Section titled “Onboarding Bootstrap Tooling Design Document”Status
Section titled “Status”In Review
Problem
Section titled “Problem”The backend services team ships to production daily and runs a shared on-call rotation. New engineers are excluded from on-call for their first thirty days, which makes the two-week window to first productive contribution load-bearing: Priya (starting Mon Jun 22, buddy Arjun, DRI Mei) needs credentials, repository access, tooling configuration, and enough codebase familiarity to ship a real change before the two-week mark.
The current setup process is unstructured. The buddy walks the new hire through access and tooling verbally on day one, which takes two to four hours of buddy time, produces no machine-readable record, and leaves access gaps undiscovered until they block work mid-week. There is also no early-warning mechanism for the failure mode where the pre-scoped week-two task has not been selected before day one, which causes week two to collapse without a visible signal.
The constraints shaping the solution space:
- Buddy capacity is fixed. The pairing protocol targets 30-40% of the buddy’s time in week one and 15-20% in week two. Tooling must reduce access-verification overhead, not add coordination cost on top of it.
- The existing
scripts/bootstrap.shruns on every local environment setup and must not be broken. The design extends it rather than replacing it. - The tooling must be operational before Jun 22. Anything requiring a multi-sprint build cycle is out of scope.
- Staging secret manager access for Priya went into the infra queue Jun 23. Her own credentials must be verifiable before the on-call alert drill on Jul 2. The tooling needs to surface this gap automatically if it persists.
Proposed Design
Section titled “Proposed Design”Access verification module
Section titled “Access verification module”A new script at scripts/check-access.sh is called at the end of bootstrap.sh. It runs each required access grant against a live endpoint and prints a result line per item:
[PASS] source-control[PASS] vpn-sso[FAIL] secret-manager-staging - open: https://it.internal/request/secret-manager[WARN] sso-propagation - SSO credentials may take up to 4h to propagate; verify manually with buddyVerification methods per item:
| Item | Method |
|---|---|
| Source control | git ls-remote git@source.example.internal:backend/services.git HEAD |
| VPN / SSO | TCP probe to vpn-healthcheck.internal:443 |
| CI/CD pipeline | GET /api/v1/pipelines?user=<email> on the CI host; expects 200 and a non-empty list |
| Observability platform | Authenticated GET /api/dashboards; unauthenticated 200 is not sufficient |
| Secret manager (staging) | vault list secret/staging with the engineer’s token |
Exit code equals the count of [FAIL] items. [WARN] lines do not affect the exit code. The IT portal URL printed on failure is derived from a static map in config/access-portal-urls.json, keyed by item ID.
Machine-readable onboarding file
Section titled “Machine-readable onboarding file”A JSON file committed to main at onboarding/priya-2026-06-22.json before Priya’s first day:
{ "engineer": "Priya", "start_date": "2026-06-22", "buddy": "Arjun", "dri": "Mei", "oncall_eligible_date": "2026-07-22", "first_ship_target": "2026-07-03", "access_items": [ { "id": "source-control", "label": "Source control org membership", "verified_by": null, "verified_at": null }, { "id": "vpn-sso", "label": "VPN credentials and SSO", "verified_by": null, "verified_at": null }, { "id": "cicd", "label": "CI/CD pipeline access", "verified_by": null, "verified_at": null }, { "id": "observability", "label": "Observability platform", "verified_by": null, "verified_at": null }, { "id": "secret-manager-staging", "label": "Secret manager read access (staging)", "verified_by": null, "verified_at": null }, { "id": "chat-channels", "label": "#backend-services, #incidents, #deployments, #team-random", "verified_by": null, "verified_at": null } ], "first_issue_id": null, "first_pr_merged_at": null, "onboarding_closed_at": null}The buddy marks items verified by running:
scripts/verify-onboarding.sh source-controlThe script writes the verifier’s username and a UTC timestamp into verified_by and verified_at for that item and commits the change. When check-access.sh passes a check automatically, verify-onboarding.sh is called by bootstrap.sh to populate that item without requiring buddy action. Items the script cannot verify (SSO propagation, chat channels) remain null and require explicit buddy sign-off.
Pre-day-one preflight gate
Section titled “Pre-day-one preflight gate”scripts/verify-onboarding.sh --preflight onboarding/priya-2026-06-22.json is intended to be run by Mei or Arjun before Jun 22. It checks:
- An issue tagged
onboarding:priyaexists in the issue tracker. - That issue carries no more than one service label (cross-referenced against
docs/service-map.json). - The issue is not tagged
oncall-risk.
If any check fails, the script exits non-zero and names the missing item. This surfaces the “task not ready on day one” failure mode before Priya arrives, not after.
CI badge wiring
Section titled “CI badge wiring”Three badges are wired to the onboarding file and the CI pipeline:
onboarding-bootstrap-passing: CI runscheck-access.shnightly against Priya’s credentials whileonboarding_closed_atis null. Badge is green when exit code is 0.oncall-week-five: Static metadata badge generated fromoncall_eligible_datein the JSON file.first-ship-week-two: Webhook triggered when a PR taggedonboarding:priyais merged. The CI bot writes the merge timestamp intofirst_pr_merged_at, closes the onboarding window by settingonboarding_closed_at, and notifies Mei in #backend-services.
The nightly check run stops automatically once onboarding_closed_at is set.
Alternatives Considered
Section titled “Alternatives Considered”Unstructured verbal walkthrough (current state). The buddy walks through access verbally on day one. Rejected: leaves no record, does not surface access gaps until they block work, and fully loads the buddy during the highest-cost part of week one without producing a verifiable artifact.
Full onboarding web dashboard with a database backend. A web UI tracking checklist state with history and a reporting view. Rejected on scope: the dashboard can be built later on top of the JSON file. A committed JSON file is verifiable, diffable, and can be operational before Jun 22. A web dashboard cannot.
Tech lead as sole onboarding owner. Remove the buddy role and have Mei run all pairing sessions directly. Rejected: concentrates the highest-cost time resource on administrative work, and removes the peer buddy relationship, which carries distinct belonging value that the DRI role cannot replicate.
Single combined bootstrap script. Fold access verification and onboarding tracking into the existing bootstrap.sh. Rejected: bootstrap.sh runs on every local environment setup. Onboarding verification runs once per engineer. Merging them tightly makes both harder to maintain and test.
Risks and Open Questions
Section titled “Risks and Open Questions”Risk: false positives from check-access.sh. Authenticated endpoint checks can pass while individual credential propagation is still in flight (SSO in particular has up to a four-hour window). Mitigation: checks that cannot verify individual credentials emit [WARN] rather than [PASS] and require explicit buddy verification in the JSON file. The warn-vs-pass distinction must be documented in the script so future maintainers do not promote warns to passes without understanding the tradeoff.
Risk: staging access not resolved before Jul 2. Priya’s staging secret manager request has been in the infra queue since Jun 23. She needs her own credential before the on-call alert drill on Jul 2. The nightly CI run will report [FAIL] secret-manager-staging and keep the badge red, creating a visible signal. The escalation path when the badge stays red - Mei escalates through the engineering manager - is a process dependency this tooling cannot enforce, only surface.
Open question: onboarding window close when first PR misses target. The design closes the window on first merge of a PR tagged onboarding:priya. If no such PR merges by first_ship_target (Jul 3), the window stays open indefinitely and CI keeps running nightly checks. Proposal: if first_pr_merged_at is still null on first_ship_target, the CI bot posts a notification to Mei and halts the nightly run pending her decision to extend or close. This needs explicit sign-off before implementation.
Open question: simultaneous onboarding of two engineers. The current design stores one file per engineer. If two engineers onboard at the same time, the nightly CI run must discover and iterate over all open onboarding files (glob onboarding/*.json where onboarding_closed_at is null). This is a straightforward extension but has not been prototyped. The immediate case (Priya only) does not require it.
Mentorship-by-Sponsorship Design Document
Section titled “Mentorship-by-Sponsorship Design Document”Written to Dana, who implemented this in 2016, and to Priya, who is currently running inside it.
Status
Section titled “Status”Accepted - implemented March 2016; actively replicated as of February 2026.
Problem
Section titled “Problem”In early 2016 my organization needed a lead for the Alderton platform migration: six months, cross-functional, visible to senior stakeholders. The organization expected someone who had already run a project at that scope. I had not.
Dana had a junior contributor - me, in my second year - who had demonstrated aptitude in smaller work but had no track record at the scale the role required. The conventional move was to name an established mid-level person. The opportunity cost of that move was that I would not develop.
The constraints shaping the solution space:
Real accountability cannot be delegated in halves. If the lead carries decisions made by someone more senior, the project delivers but the person does not develop. The accountability has to land somewhere and stay there.
Simulated stakes do not transfer. Practice projects and pre-assignment coaching produce a kind of readiness that does not survive first contact with a real deadline and a stakeholder expecting answers. The capacity for judgment under pressure can only be built under actual pressure.
The sponsor holds an unrecoverable position. If the project failed, the organization would remember Dana’s judgment, not my inexperience. The asymmetry between candidate exposure and sponsor exposure is not incidental - it is the central structural fact of this design.
Co-lead arrangements teach nothing at the relevant margin. When a decision becomes difficult, the senior person absorbs it. The junior person watches. That is observation, not development.
Proposed Design
Section titled “Proposed Design”Dana’s implementation can be characterized as three sequential phases with a governing constraint that operates across all three.
Phase 1 - Nomination
Nominate the candidate before they feel ready. Do not wait for the candidate to ask. Self-nomination would allow them to frame the appointment as something they earned through confidence. Confidence is not a prerequisite. Submit the name to the relevant stakeholder without flagging the appointment as developmental stretch. The candidate is the lead, not a developmental candidate.
Interface: Sponsor names the candidate to the decision-maker unilaterally. Candidate learns they have been nominated, not asked.
Phase 2 - Close shadowing (months 1 through 3)
Stay visible and available. Attend key stakeholder meetings for the first two months and say little. When the candidate asks a question, answer it - then ask a sharper one back, the kind that surfaces what they are not yet seeing. When the candidate gets something wrong in a meeting, correct it privately afterward, not during.
Critical negative constraint: Do not accept the work back. If the candidate offers to return accountability in a moment of panic, decline without softening the boundary. Accepting the work back ends the intervention.
State transition signal: The candidate runs a stakeholder meeting without referencing the sponsor for validation. At that point, advance to Phase 3.
Phase 3 - Withdrawal (months 3 through 6)
Reduce visibility. Stop attending standing meetings. Remain available by request but do not initiate contact. The work is the candidate’s.
Exit condition: The project delivers with the candidate as the named accountable party. The sponsor does not appear in the post-mortem as a contributor.
Governing constraint (all phases): The sponsor must hold back intervention even when the candidate is working more slowly or less cleanly than the sponsor would. This is not a feature of the design - it is the design. Every accelerated intervention shifts attribution of the outcome away from the candidate. The candidate will register this transfer even if neither party names it.
Alternatives Considered
Section titled “Alternatives Considered”Assign an established mid-level lead. Delivers the project. Does not develop the junior contributor at the relevant scope. The safer call and the costlier one over time.
Co-lead arrangement. Grants the candidate title and nominal shared accountability. When a call becomes difficult, the senior person absorbs it. The junior person observes. The candidate learns that accountability is shared when it is easy and transferred when it is hard. That is the wrong lesson.
Intensive pre-assignment coaching. Better than no preparation. Does not transfer. The stakes remain simulated and the candidate knows it. Coaching produces confidence without the evidence base that makes confidence durable.
Wait for a fuller track record at smaller scope. Each deferred cycle is another period the candidate spends at a scope that is not building the capacity you need them to have. Deferral compounds in cost. At sufficient lag, the candidate is developed by someone else or stops waiting.
Risks and Open Questions
Section titled “Risks and Open Questions”Sponsor reputational exposure is real and unhedged. Dana held a vulnerable position for the full duration of the Alderton project. There is no mitigation that preserves the intervention’s value while removing this exposure. A sponsor who cannot hold this position should not use this design.
The nomination decision is the critical gate. The intervention has no clean early-exit mechanism. If the sponsor has misread the candidate’s baseline, taking back accountability midway through invalidates the development value and leaves the project in a worse position than if it had been led by the established mid-level person from the start.
The patience cost is not visible on delivery plans. The time Dana spent absorbing my uncertainty across six months - check-ins, redirects, holding space while I worked through decisions she could have made in an afternoon - does not appear in any project management view. Organizations will not have priced this overhead when the intervention is running. Sponsors should expect to carry it without recognition during the project.
Replication fidelity degrades without a written record. When I nominated Priya Osei for the Cassava data-pipeline rebuild in February 2026, I was working from memory. I had one pattern to replicate and I had experienced it from the inside. Someone who had not experienced this approach would have no implementation to work from. That is the reason this document exists.
Appendix
Section titled “Appendix”From the 2016 record (Alderton platform migration):
- March 2016: Dana submits nomination for Sable Marchetti as project lead
- Months 1 through 2: Dana attends primary stakeholder review meetings; intervenes rarely
- Week 11: Sable offers to hand the project back; Dana declines
- Month 4: Dana stops attending standing meetings; available on request
- April 2017: Project closes; Dana not named in the post-mortem
From the 2026 record (Cassava data-pipeline rebuild):
- February 2026: Priya Osei nominated for project lead; she said she was not ready
- Week three, March 2026: Priya stuck on handoff logic; Sable sits on her hands
- June 2026: Priya is four weeks ahead of original schedule
Note on this document: The 2016 intervention was not documented at the time. This record was written in June 2026, ten years after implementation, by the person who was developed by it. The system Dana built in 2016 is currently running in my team. I did not have a name for what she did until I found myself doing it. The debt is acknowledged. The design is worth preserving.
Rest Window Expansion Design Document
Section titled “Rest Window Expansion Design Document”Status
Section titled “Status”Draft
Problem
Section titled “Problem”The current rest window - phone in a drawer from Saturday at 8 p.m. through Sunday at 6 a.m. - is ten hours. It has been held for two consecutive weeks, and the data from those weeks is useful: a ten-hour window is long enough to produce a change in Monday’s thinking, but the edges are losing most of the value.
Two failure points are visible.
The entry boundary starts too late. Saturday at 8 p.m. follows a full Saturday that runs on the same monitoring patterns as a workday. Rest begins after the readiness to rest has been eroded. The first hour or two of the window are spent in decompression rather than rest.
The exit boundary is too abrupt. When the phone comes back Sunday at 6 a.m., the first fifteen minutes involve scanning everything - inbox, notifications, ticket tracker, message queue. Whatever steadiness built up during the night compresses before the next week has started. The rest is ending before the rest day is over.
The next design must address both edges without introducing a third problem: expanding the window past the point where it can be held, or building an architecture so complex it becomes a productivity system in disguise.
Constraints:
- The window must anchor to recurring calendar signals that exist regardless of week content (sunset, morning, established meal times).
- Rules must be specifiable in advance, before the moment where rationalizing a check becomes easy. In-the-moment rule-setting fails.
- The pre-rest and post-rest protocols must complete in under fifteen minutes each. Longer protocols become preparation work, which defeats their purpose.
- The productivity accounting habit - the background process that tallies whether the rest was worth it - does not respond to direct attack. Any design that requires suppressing that habit as a precondition will fail.
Proposed Design
Section titled “Proposed Design”Window boundaries
Section titled “Window boundaries”The new window runs from Friday at sundown through Sunday at 7 p.m.
Sundown is used as the Friday start rather than a clock time because it shifts with season. The chosen signal is: close the laptop and put the phone in the dedicated drawer when the light through the west-facing window changes. This is a physical cue, not a calendar notification.
Sunday at 7 p.m. is chosen over Sunday morning because the status data shows the re-entry compression happens when the window closes before the rest day is done. Ending at 7 p.m. preserves the full day and allows a deliberate transition rather than an abrupt one.
Physical protocol
Section titled “Physical protocol”The phone-in-drawer protocol extends to cover the laptop. Both devices go into the same physical location - the top drawer of the desk - not just out of reach but visually unavailable. The drawer is closed. One device exception: the household tablet is kept available for non-work use (reading, navigation, family coordination). It is not logged into work accounts and does not have notifications enabled for work systems.
Pre-rest log (five minutes, Friday at sundown)
Section titled “Pre-rest log (five minutes, Friday at sundown)”Before the window opens, write the following by hand, not typed:
- What I expect to feel in the first two hours.
- What I am most likely to rationalize as a necessary check.
- The rule that governs that check - what I will do instead if the urge arrives.
This log is not reviewed during the rest period. It is read Sunday evening during the post-rest review to compare expectation against what actually happened.
Post-rest review (fifteen minutes, Sunday at 7 p.m.)
Section titled “Post-rest review (fifteen minutes, Sunday at 7 p.m.)”When the window closes:
- Read the pre-rest log.
- Write two to three sentences on what happened versus what I expected.
- Note one thing that helped and one thing that worked against the rest.
- Cap the inbox scan at thirty minutes: no replies, triage only.
No productivity accounting. The purpose of the post-rest review is observational, not evaluative. The difference: observational asks “what happened?” Evaluative asks “was it worth it?” The second question is the one to block.
Component summary
Section titled “Component summary”| Component | Trigger | Duration | Rule |
|---|---|---|---|
| Window open | Friday sundown (light cue) | - | Laptop and phone to drawer |
| Pre-rest log | At window open | 5 min | Write by hand, three prompts |
| Window close | Sunday 7 p.m. | - | Devices out of drawer |
| Post-rest review | At window close | 15 min | Read log, write observations |
| Inbox scan | After post-rest review | 30 min cap | No replies, triage only |
Alternatives Considered
Section titled “Alternatives Considered”Extend the current Saturday window incrementally. Move the Saturday start from 8 p.m. to 6 p.m., then 4 p.m. in subsequent weeks. This solves the late-start problem without a structural change. It does not solve the Sunday re-entry compression. Rejected because two of the three identified failure points remain, and incremental extension has not moved the window in five weeks of trying.
Add a mid-week half-day instead of extending the weekend window. A Wednesday morning or afternoon window would distribute rest across the week rather than concentrating it at the weekend. Rejected because the two failure points are both boundary problems specific to the weekend window; a mid-week window introduces a new design without fixing the existing one. The existing window needs to work before a second one is added.
Use app-blocking software to enforce the boundary. Scheduled app blockers on the phone and laptop could enforce the window without relying on the physical drawer protocol. Rejected for two reasons: the physical cue creates a different relationship to the constraint than a software limit (software is defeatable with a tap and a reason); and the pre-rest log and physical protocol together are the behavior being designed, not just the absence of checking. Enforcement software would implement the absence without the behavior.
Use a shorter, more frequent window. One hour of complete silence each morning, every day. This is an entirely different practice and is worth designing separately. Rejected here because the data from weeks 1 through 14 shows that the quality change in Monday’s thinking comes from a sustained break, not from accumulating shorter ones.
Risks and Open Questions
Section titled “Risks and Open Questions”The productivity accounting habit will survive the new design. The background process of tallying whether the rest was worth it is the largest identified risk. The post-rest review is designed to be observational rather than evaluative, but the accounting habit does not respond to design constraints - it runs underneath them. Working hypothesis: it resolves over time as evidence accumulates. The pre-rest log is intended to build that evidence base. Risk level: high. Mitigation: name the habit explicitly in the pre-rest log each week; track whether the observation changes over weeks.
Friday evening carries different scheduling pressure than Saturday evening. Friday often carries end-of-day processing that does not exist on Saturday. The sundown start may conflict with finishing commitments that have natural Friday deadlines. Mitigation: the rule is “sundown, unless a specific named commitment extends past sundown.” The named commitment must be written in the pre-rest log. Unwritten commitments default to the window.
What counts as rest within the window is still undefined in some areas. Reading, walks, and cooking are clearly rest. Whether journaling counts as rest or as work-adjacent reflection is a judgment call that varies by day. Open question for the first three weeks: track whether each activity left the rest window feeling more or less intact. Do not legislate the answer in advance.
The post-rest review may become the productivity accounting habit in a different form. If the review expands or becomes increasingly analytical, it will recreate the problem it was designed to interrupt. Mitigation: the review has a hard cap of fifteen minutes and three fixed prompts. If the impulse to add to the review format appears, log it in the pre-rest log the following week rather than acting on it.
Open question: whether the extended window changes Monday availability perceptibly. The ten-hour window did not draw external comment. A forty-plus-hour window is a different surface area. Observe for the first four weeks before treating this as a design concern.
Knowledge Continuity System Design - Howard Thayer Departure
Section titled “Knowledge Continuity System Design - Howard Thayer Departure”Status
Section titled “Status”Accepted
Problem
Section titled “Problem”Howard Thayer is retiring from Crestfield Group on June 27, 2026, after twenty-six years as Operations Coordinator. The knowledge transfer problem this creates is not routine. Howard did not accumulate scope or headcount over his tenure; he accumulated context. The gap is structural, not positional.
The constraints shaping the solution space:
Time bound: Structured sessions with Howard were limited to May and June 2026. Two sessions are complete. His last day is June 27. No extension is possible.
Knowledge type: The majority of Howard’s value was procedural and relational - how to navigate a vendor dispute, which contacts respond to which communication style, how to read an ambiguous alert correctly. These do not extract cleanly into reference documentation.
Capacity: The team cannot assign a full-time documentation effort. Dana Reyes and Marcus Okonkwo have operational workloads that preclude more than one weekly session each.
Freshness risk: Documentation created under time pressure tends to go stale quickly if not embedded into active operational workflows. A one-time knowledge dump is likely to be correct at creation and wrong by Q4 2026.
Coverage asymmetry: What Howard knows he knows is documentable. What he knows without knowing he knows it is not. The second category may be larger than the first.
Proposed Design
Section titled “Proposed Design”The system has four components. Each targets a distinct knowledge type and has a defined owner, delivery format, and location.
Component 1 - Incident-Response Runbook
Section titled “Component 1 - Incident-Response Runbook”Owner: Dana Reyes
Delivery: docs/ops/incident-response.md
Coverage: Vendor escalation paths, four utility contacts not present in any official directory, automated alert interpretation heuristics, sequence-of-events decision tree for ambiguous incidents
Structure:
docs/ops/incident-response/ vendor-escalation/ tier-1-contacts.md # Howard's four utility contacts, with per-contact notes escalation-thresholds.md # When to escalate vs. resolve internally alert-interpretation/ ambiguous-alerts.md # Patterns Howard recognized that monitoring does not surface incident-sequence/ initial-assessment.md # First 15 minutes cross-team-coordination.mdThe runbook captures process, not judgment. The judgment gap is acknowledged in Risks below.
Component 2 - Mentee Practice Archive
Section titled “Component 2 - Mentee Practice Archive”Owner: Carolyn Marsh
Delivery: team/former-mentees.md
Coverage: Howard’s observable coaching behaviors - how he framed problems for people who were panicking, the question sequences he used to surface problems without stating them directly, his practice of absorbing a situation before speaking
Structure:
team/mentee-archive/ practices/ problem-framing.md # Sourced from: Priya Sandhu, Ben Holter question-sequencing.md # Sourced from: Marcus Okonkwo pre-response-pause.md # Named and described by four contributors critical-incidents/ [year-event-slug].md # One file per incident recalled in contributor notesContributors: Priya Sandhu, Ben Holter, Marcus Okonkwo, and four additional colleagues. Contribution method: written notes submitted by June 24. Editorial pass by Carolyn Marsh to remove eulogy framing and retain practice-level specificity.
Component 3 - Vendor and Contact Transfer
Section titled “Component 3 - Vendor and Contact Transfer”Owner: Three named successors (listed in team/contact-owners.md)
Delivery: Contact records updated in the internal CRM by June 20
Coverage: All contacts Howard maintained personally, with ownership explicitly reassigned
Transfer is complete. No contacts in the CRM list Howard as sole owner. Relationship quality is not transferable; the contacts exist and will be owned by named humans. That is the best achievable state.
Component 4 - Send-Off Event
Section titled “Component 4 - Send-Off Event”Owner: Carolyn Marsh Date: June 25, 2026 Format: In-person and remote hybrid; recorded for regional sites that could not send a representative Duration: 90 minutes allocated. Howard spoke for four minutes.
The send-off serves two functions the other components cannot: it creates a shared organizational timestamp for the departure, and it allows Howard to speak in his own voice. Both matter. Neither is documentable in retrospect.
Alternatives Considered
Section titled “Alternatives Considered”Documentation sprint (rejected): A concentrated 40-hour extraction effort in May 2026, with Howard in the room full-time. Rejected for three reasons: the output would be dense and hard to maintain; Howard’s knowledge is not structured as documentation - forcing it into that form produces a document nobody reads; 40 hours of his time in his final weeks is not the right use of that time.
Formal mentoring assignment (rejected): Assigning Howard a “Mentor” title for his final six months and creating a structured program. Rejected because this title has historically read as sidelining at Crestfield rather than recognition, and because formalizing his mentoring function would have changed its nature. Howard mentored by working alongside people, not above them. Formalization would have ended that.
Recording all sessions (rejected): Video or audio recording the knowledge sessions with Dana Reyes and Marcus Okonkwo. Rejected because recordings are not searchable and are typically not watched under pressure. The marginal value over structured written notes was assessed as low. The interpersonal cost of being recorded was assessed as non-trivial for sessions requiring candor.
Delay retirement date (not pursued): Not considered a real option. Howard had set his date. The system was designed to the constraint, not to override it.
Risks and Open Questions
Section titled “Risks and Open Questions”Tacit knowledge gap (accepted risk): The runbook captures Howard’s documented decision sequences, not his judgment under genuine ambiguity. The team will not know the full shape of this gap until the first novel incident he is not available for. No mitigation exists beyond naming it. The risk is real and will surface on its own schedule.
Runbook staleness: The runbook is accurate as of June 2026. Operational context changes. Target: quarterly review against actual incident patterns. Owner: Dana Reyes. First review: September 30, 2026. There is no enforcement mechanism beyond the calendar item; this is a known gap in the design.
Morale lag: Howard’s absence will become concrete six to eight weeks after his last day, when the reflex to call him is still present but he is not. No structural mitigation is planned. Open question: should there be? Managers with Howard’s former mentees should check in during July and August. Whether that constitutes sufficient coverage is unknown until tested.
Archive discoverability: The mentee archive is only as useful as its findability under pressure. If no one searches it during an incident, it provides no value at the moment it is most needed. Open question resolved as of June 27: add cross-reference links from the incident-response runbook to the mentee archive before July 15. Owner: Carolyn Marsh.
Contribution completeness: Howard contributed through knowledge he held without knowing he held it. By definition, the capture is incomplete. The runbook and archive represent what was recoverable in the available time. This is a known residual, not a failure of the process. It should be named as such in the runbook’s own header.
Checkout Rebuild: Parallel-Run Architecture Design
Section titled “Checkout Rebuild: Parallel-Run Architecture Design”Status
Section titled “Status”Draft - January 2025
Problem
Section titled “Problem”The production checkout system cannot be safely modified. It has accumulated five years of emergency patches, carries no meaningful test coverage, and couples session state to payment logic in ways that are partially undocumented. Two prior attempts to refactor it in place both stalled: the risk surface was too wide to contain, and both teams eventually cut losses rather than ship a regression.
The consequence is visible in the metrics. Cart abandonment has been elevated for three years. Instrumentation points to the payment flow as the primary failure zone, but improving the payment flow requires touching the session layer, and touching the session layer without adequate test coverage is how the system arrived at its current state.
The constraints that shape the solution space:
- No full-downtime migration window. Checkout runs continuously. A planned outage long enough to support a cutover requires board-level approval, and that path is closed for this project.
- No in-place refactor. Two teams established that the existing coupling prevents safe incremental modification at any reasonable pace. A change to one subsystem creates regression surface in another; the coverage to protect against that regression does not exist.
- Must be validatable under real load before full cutover. Staging environments do not reproduce the edge cases that live sessions generate at volume. The only meaningful test of a checkout system is real checkout traffic.
- Must preserve rollback capability throughout. Any architecture that eliminates fallback to the current system before the new one is validated is unacceptable given the revenue exposure.
These constraints eliminate the in-place refactor and the big-bang cutover as viable paths. The remaining design space is a parallel-run approach: build a new pipeline alongside the current one, route real traffic to it incrementally, and decommission the old system only after sufficient validation under production conditions.
Proposed Design
Section titled “Proposed Design”Component Boundaries
Section titled “Component Boundaries”The new checkout is deployed as a separate service (checkout-v2). A traffic router at the session boundary determines which pipeline handles each session. The legacy system (checkout-v1) remains live and on-call throughout the migration.
Client | v[Session Gateway] (unchanged) | +--[Traffic Router]--+---> checkout-v2 (new pipeline) | | +-> checkout-v1 <---+ (legacy, fallback) | | | +--[Session Store (shared)]--+The Session Gateway is the only public entry point and is not modified. The Traffic Router reads a routing key from session metadata to decide which pipeline receives the request. Both pipelines read from and write to a shared Session Store during the parallel period.
Traffic Routing
Section titled “Traffic Routing”Routing is cohort-based on a hash of user_id mod N, where N is a configurable shard count. This ensures that:
- A given user routes consistently to the same pipeline for the duration of their session.
- Rollback for a cohort is a configuration change, not a code deploy. Recovery time from a routing-level rollback is under five minutes.
- Coverage expands by widening the shard range in config, without a service deployment.
Session Schema Contract
Section titled “Session Schema Contract”Both pipelines must read and write a compatible session format during the parallel period. All v2 extensions are namespaced under v2_ext so that v1 reads are unaffected by keys it does not recognize.
{ "session_id": "...", "cart_id": "...", "checkout_state": "...", "v2_ext": { "pipeline_version": "v2", "routing_shard": 42, "step_entered_at": "2025-03-01T14:22:00Z" }}The v2_ext key is absent for v1-routed sessions. The router populates it at session creation for v2-routed sessions. v2 must not remove or rename any field that v1 reads; removals require a compatibility shim reviewed before decommission.
Observability
Section titled “Observability”Both pipelines emit to a shared metrics namespace so that A/B comparison is possible at the dashboard level. Required instrumentation per pipeline:
- Step completion rate per checkout step, tagged by pipeline version
- Payment authorization latency (p50, p90, p99)
- Cart abandonment rate per session cohort
- Error rate per step, tagged by error class and pipeline version
A shadow comparison job runs on a 5-minute cadence and emits a divergence metric when v1 and v2 behavioral metrics differ by more than a configurable threshold for the same cohort. This is the primary signal for gate decisions.
Rollout Phases
Section titled “Rollout Phases”| Phase | Live traffic coverage | Gate to advance |
|---|---|---|
| Shadow | 0% (replaying v1 events) | No behavioral regressions over 7-day window |
| Canary | 1-5% of sessions | Cart completion rate within 2% of v1 for same cohort |
| Ramp | 5% to 80% | No divergence alerts; at least one peak-load period covered |
| Cutover | 100% | Two peak-load periods without a rollback trigger |
Advancing between phases requires explicit sign-off; there is no automatic promotion. Each gate decision is recorded.
Decommission Criteria
Section titled “Decommission Criteria”Decommission of checkout-v1 begins only after:
checkout-v2has handled two peak-load periods at 100% traffic without a rollback event.- All session-compatibility shims have been reviewed for removal safety.
- A 31-day archive window for v1 logs and session state has elapsed.
Decommission is a separate milestone and is not in scope for this design document.
Alternatives Considered
Section titled “Alternatives Considered”In-place refactor with incremental coverage. Build test coverage incrementally while modifying the existing system. Rejected because two prior attempts established that the existing coupling defeats this approach at any reasonable pace. The coverage required to make in-place changes safe would take 18 or more months to accumulate, during which the system remains fragile. This path also does not satisfy the “validatable under real load” constraint.
Big-bang cutover. Build a new system, validate it in staging, and cut over in a single deployment event. Rejected because staging does not reproduce the edge cases that emerge at production session volumes, a failed cutover at peak load has no fast rollback path, and the revenue exposure of an emergency rollback during a checkout outage is unacceptable. This approach requires one shot to work; the parallel-run approach does not.
Risks and Open Questions
Section titled “Risks and Open Questions”Session compatibility across the migration boundary. The shared Session Store creates a living contract between two independently evolving codebases. If v2 modifies the session schema in a way that breaks v1 reads, a rollback could corrupt in-flight sessions. Mitigation: strict namespace isolation for all v2 extensions; integration tests that confirm v1 can round-trip any session written by v2.
Dual on-call surface. Running two live checkout pipelines doubles the on-call surface for the duration of the parallel period. There is no architectural way to eliminate this overhead; it is inherent to the parallel-run approach. The team needs to plan on-call rotations and runbook coverage for both systems for approximately fourteen months.
Cohort edge cases at session boundaries. Users who switch devices mid-session, share cart links, or enter checkout through deep links may land in an unexpected pipeline if the routing key is session-based but the entry point is link-based. These paths have not been fully mapped. They must be cataloged before the Canary phase advances.
Shadow comparison blind spots. The shadow phase divergence metric is only as good as the events that v1 produces. The legacy codebase has behavioral paths that do not emit structured events. Before treating shadow-phase results as conclusive, we need an audit of v1 event coverage.
Peak-load validation timing. The rollout gates require two peak-load periods at full traffic before decommission can begin. The peak calendar is not in engineering’s control. If the migration reaches the Cutover phase during a low-traffic window, the decommission gate cannot advance until the next natural peak. This may extend the parallel-run period past the planned fourteen months.
Callback and session-store ordering under concurrent sessions. The payment processor callback and session-store write are currently assumed to be ordered correctly in the legacy system, but the ordering is not enforced in code. In v2, this ordering must be explicit. If it is not, concurrent mobile and desktop sessions accessing the same cart could produce inconsistent state. This needs to be resolved in the v2 payment callback handler design before the Canary phase.
Anchor-Day Hybrid Policy - Implementation Design Document
Section titled “Anchor-Day Hybrid Policy - Implementation Design Document”Status
Section titled “Status”In Review
Problem
Section titled “Problem”The organization has adopted a structured hybrid work arrangement (ADR-0012): two mandatory anchor days per week (Tuesday and Thursday) for all non-remote-designated employees, with the remaining three days fully flexible.
The ADR records the decision. This document works through how to implement it as an operational system - the policy schema, the components that must interoperate, the exception approval workflow, and the tracking mechanism needed to keep anchor days from eroding into suggestions.
Three constraints shape the implementation space:
Dual-track authority gap. Facilities and HR are running independent workstreams on different timelines. Facilities has already communicated a room-booking policy premised on five-day potential attendance. HR has not yet published guidance. These two systems must interoperate before the all-hands announcement on July 3. Currently, they will contradict each other on that date.
Enforcement drift as the primary failure mode. ADR-0012 named this explicitly: the risk is not defiance, it is quiet erosion. Anchor days become suggestions when no one tracks attendance and managers do not hold the line. An implementation that relies entirely on manager discretion will not hold past the first quarter.
Unknown exception volume. A non-trivial number of employees hold remote-eligible designations, documented medical accommodations, or relocation arrangements made before the policy existed. The exception approval path must be defined before July 3, not retrofitted under pressure once requests start arriving in volume.
Proposed Design
Section titled “Proposed Design”Policy schema
Section titled “Policy schema”Each employee record carries three implementation-relevant fields:
| Field | Values | Authoritative owner |
|---|---|---|
location_type | anchor-required / remote-eligible / pending-review | HR; set at hire, updated by manager + people team |
anchor_exception_status | none / approved / expired | Exception workflow |
anchor_compliance_period | Current quarter | Resets quarterly |
Employees with location_type: remote-eligible are excluded from anchor day requirements. All others default to anchor-required. The pending-review state covers new hires during their first two weeks and employees whose role classification is under active review.
Components
Section titled “Components”HR system (authoritative source). Owns location_type for all employees and all exception records. Publishes a daily export to the room-booking system so attendance expectations reflect current headcount and exception status. This export is the primary interface between the Facilities and HR workstreams.
Room-booking system (Facilities-owned). Must ingest the daily HR export to calculate expected anchor-day attendance by building and floor. Must stop operating on a five-day attendance assumption before July 3. Reads anchor_exception_status to correctly handle rooms booked by or for exempt employees on anchor days.
Manager dashboard (people team-owned). Displays per-direct-report anchor compliance for the current quarter. Aggregates to trend data (consistent, inconsistent, never) rather than individual session logs - the purpose is drift detection, not surveillance. Visible to the manager and people team; not visible to skip-levels by default.
Exception workflow. Employee submits through the HR portal with a reason category (role reclassification, medical, caregiver, relocation, other). Manager reviews and approves or escalates within five business days. People team countersigns. Exceptions are time-bounded at one quarter maximum and must be renewed. Expired exceptions revert anchor_exception_status to none automatically, with a five-business-day notification before expiry.
Data flows
Section titled “Data flows”HR system --> daily export --> Room-booking system (headcount + exception flags) --> read API --> Manager dashboard (compliance status per employee) --> exception records (read/write for exception workflow)
Exception workflow --> status update --> HR system: anchor_exception_status field --> notification --> Manager dashboard: exception flag on affected employeeFacilities-HR interface contract
Section titled “Facilities-HR interface contract”The daily export format and field definitions for location_type and anchor_exception_status must be agreed between Facilities and HR before July 3. The schema above proposes the fields; both teams should ratify the names and values. Neither team should replicate the other’s authoritative data - the export is a read, not a copy.
Alternatives Considered
Section titled “Alternatives Considered”Manager discretion only (no tracking infrastructure). Trust managers to enforce anchor days and handle exceptions informally, with no central system. Rejected because ADR-0012 already named enforcement drift as the primary failure mode. A system with no visibility into compliance cannot detect drift until it is already complete. Compliance becomes a function of individual manager relationships, which resolves inconsistently across the organization.
Extend the existing Facilities badging system. Use badge-swipe data from the current system, which was built for five-day attendance tracking, to measure anchor day presence. Rejected because the badging system has no integration with HR’s location_type field. It would record swipes without knowing which employees are anchor-required versus remote-eligible, producing compliance data that cannot be acted on and cannot handle exceptions accurately.
Self-reported attestation. Employees self-report attendance via the HR portal; managers confirm. Rejected because self-attestation with manager confirmation adds friction for compliant employees and is trivially gamed by non-compliant ones. It also concentrates the enforcement burden on managers, which reinforces the drift risk rather than counteracting it.
Risks and Open Questions
Section titled “Risks and Open Questions”Facilities-HR authority gap (high, before July 3). If the room-booking system is not updated to reflect anchor-day headcount expectations before the all-hands announcement, employees will encounter booking constraints that contradict the policy they just heard. This is the most immediate risk and requires a named decision owner by June 28 - the date of the executive sponsor briefing.
Exception volume spike at launch. The workflow above assumes steady-state volume. A significant proportion of the workforce submitting exceptions in the first two weeks will produce a backlog under the five-business-day review window. The people team should define triage criteria before July 3 and communicate expected response times to employees when the policy is announced.
Quarterly reset behavior for standing accommodations. Exceptions expire at the quarter boundary and must be renewed. Employees with medical or caregiver accommodations that will not change quarter to quarter face recurring renewal overhead. Open question: should long-term accommodations be renewable annually, with the quarterly reset applying only to role-classification and relocation exceptions?
Manager dashboard timing. The dashboard does not need to launch on July 3 alongside the policy, but delay creates a blind spot. If the dashboard ships after Q3 begins, it should cover the full quarter retroactively, or the first quarter will have no drift signal. If it cannot be retroactive, the launch date should be treated as the start of the first measured quarter and communicated to managers explicitly.
New hires after July 3. The Manager FAQ (target: June 27) should include an onboarding section covering anchor day expectations. New hires who arrive after the policy is live will infer expected behavior from peer behavior, which may already be drifting by the time they join.
Appendix
Section titled “Appendix”Field value definitions
Section titled “Field value definitions”location_type: anchor-required - Employee works from a geography with office access and has no approved exception. Expected on-site Tuesdays and Thursdays.
location_type: remote-eligible - Role was designated remote at hire, or has been formally reclassified by manager and people team. No anchor day obligation.
location_type: pending-review - Classification is under active review, or employee is within their first two weeks. Anchor day expectation is advisory during this state.
anchor_exception_status: approved - Active approved exception. Duration and reason category are stored in the exception record; the field itself carries only the status.
anchor_exception_status: expired - Exception period lapsed without renewal. System reverts to anchor-required behavior. Manager and employee receive notification five business days before expiry.
Tidemark Feedback Ranking Engine - Design Document
Section titled “Tidemark Feedback Ranking Engine - Design Document”Status
Section titled “Status”Accepted
Problem
Section titled “Problem”Small product teams accumulate customer feedback across support tickets, chat threads, survey exports, and notes from customer calls. The existing workflow to consolidate that scatter into a ranked roadmap is manual: read everything, copy items into a spreadsheet, and argue about priority in a meeting. The goal of Tidemark is to eliminate that step.
The ranking engine is the core subsystem. It must:
- Accept normalized feedback items from multiple source connectors (the ingestion layer is designed separately; this doc covers ranking and output only).
- Group items by recurring theme without requiring users to define themes upfront.
- Score each theme so the output order is defensible to a product team - not just algorithmically consistent, but intuitively correct for a team comparing what customers asked for this month versus six months ago.
- Produce a shareable read-only view that a team can send to stakeholders without requiring stakeholder sign-in.
Constraints that shape the solution space:
- Target teams are 2-20 people. The output must be usable without data science expertise.
- Input volume ranges from dozens to a few thousand items per sync depending on team size and how long feedback has been accumulating. The engine must handle both extremes with the same code path.
- The CLI runs locally (
tidemark sync). Ranking may call a remote service for theme extraction, but the core scoring step must work without a persistent server. - The shareable view (
tidemark share) is a static URL. It cannot require authentication on the reader side.
Proposed Design
Section titled “Proposed Design”Data model
Section titled “Data model”A normalized feedback item (produced by any source connector) carries:
FeedbackItem { id: string // stable across syncs source: string // connector id: "zendesk", "csv", "slack-channel" text: string // raw feedback text, max 4000 chars created_at: ISO8601 // original creation date, not import date tags: string[] // optional; connector may pass through existing labels}A theme groups one or more feedback items:
Theme { id: string // content-hash of the canonical phrase label: string // 3-7 word summary generated at extraction time items: string[] // FeedbackItem ids score: number // computed by the scoring step recency: ISO8601 // created_at of the most recent item in this theme}Theme extraction
Section titled “Theme extraction”Theme extraction is a remote call to a text-grouping endpoint. The input is a batch of FeedbackItem texts; the output is a mapping from item id to theme label. The local client:
- Sends items in batches of 200.
- Deduplicates by content hash before sending; items already present in the local store skip the extraction call entirely.
- Writes extraction results to a local SQLite store (
~/.tidemark/store.db) so subsequent syncs only send new items.
The extraction step does not assign scores. It returns labels and groupings only.
Scoring
Section titled “Scoring”Scoring runs entirely locally after extraction returns. The score for a theme is:
score = frequency_weight * log(item_count + 1) + recency_weight * decay(days_since_most_recent_item)Default weights: frequency_weight = 0.6, recency_weight = 0.4. These are configurable in .tidemark/config.toml but ship with the defaults.
The decay function returns 1.0 for items created within the last 30 days and decreases to 0.0 at 365 days. The curve is linear.
Scores are recomputed on every sync. The local store caches item counts and recency values per theme so that scoring does not re-read all items.
Shareable view generation
Section titled “Shareable view generation”tidemark share produces a static JSON snapshot of the current ranked theme list and uploads it to a Tidemark-hosted CDN bucket. The command returns a URL of the form:
https://tidemark.io/view/<token>The token is a 16-character base62 string derived from a hash of the workspace ID and the sync timestamp. Views are immutable: a new sync produces a new URL; the old URL continues to work until it expires (default: 90 days).
The view page is a pre-built static shell that fetches the JSON snapshot at load time. Readers do not need a Tidemark account. The shell is hosted at tidemark.io/view/* and served from the same CDN.
Component boundaries
Section titled “Component boundaries”CLI (local) | +-- SourceConnectors[] <- ingestion; designed separately | +-- ExtractionClient <- remote call; batches FeedbackItems, returns Theme labels | +-- LocalStore (SQLite) <- caches items, themes, extraction results | +-- ScoringEngine <- runs locally; reads from LocalStore; writes scored Theme list | +-- ShareClient <- uploads JSON snapshot; returns view URLThe CLI is the only entry point. There is no agent, daemon, or background sync.
Alternatives Considered
Section titled “Alternatives Considered”Scoring on the server, not the client
Section titled “Scoring on the server, not the client”An earlier design computed scores server-side and returned a pre-ranked list alongside extraction results. This simplified local code but introduced a round-trip dependency for every sync and required the server to know each team’s weight configuration.
Scoring moved to the client because the inputs (item counts, recency values) are already available locally after extraction, the computation is cheap, and keeping it local makes the weight configuration transparent. Teams can inspect and modify .tidemark/config.toml without trusting a server to honor it.
Embedding-based clustering instead of a text-grouping API
Section titled “Embedding-based clustering instead of a text-grouping API”An earlier spike used local sentence embeddings to cluster feedback before ranking. Clustering quality was acceptable for batches of a few hundred items but degraded on short support-ticket text (under 15 words) and required shipping a 200 MB model file with the CLI package.
The text-grouping API produces more consistent labels at short text lengths and removes the model weight from the install. The tradeoff is a remote dependency during extraction. Items extracted on a previous sync are never re-sent, so the dependency is absent for teams who sync daily against a stable feedback corpus.
Mutable shareable views (live-updating URLs)
Section titled “Mutable shareable views (live-updating URLs)”An earlier design updated the view JSON in place on each sync, so the same URL would reflect the latest rankings. This failed in user testing: stakeholders who bookmarked a link expected it to represent a moment in time, not a live feed, and two stakeholders in the same meeting were looking at different states of the same URL.
Immutable snapshots with new URLs on each sync resolved the ambiguity. The old URL remains accessible, which is useful for comparing how priorities shifted between syncs.
Risks and Open Questions
Section titled “Risks and Open Questions”Extraction API availability on first sync. The first sync for a new team sends all historical items. A team with years of accumulated feedback in a high-volume ticket tracker could send tens of thousands of items. The batch architecture handles this at the API level, but first-sync duration is unbounded from the user’s perspective. Risk: the sync looks hung. Mitigation: progress output in the CLI (tidemark sync --verbose) and a clear completion signal. No timeout is applied to first syncs.
SQLite concurrency. The local store serializes writes. If a user runs tidemark sync and tidemark share simultaneously, the share command may read a partially-updated store. Current mitigation: the CLI prints a warning if the store is locked and exits rather than reading a partial state. Open question: whether a file lock is needed or whether the warning is sufficient for the target user base.
Recency weight calibration. The default weights (frequency: 0.6, recency: 0.4) were set based on feedback from the twenty-two teams in the early-access cohort. That cohort validated that the ranked output matched their intuition, but skewed toward teams with active support volumes. Teams that generate feedback in infrequent bursts - quarterly surveys, annual customer calls - may find the recency decay too aggressive. Open question: expose a decay half-life setting in config, or add a --recency-mode flag to sync.
View expiry. The 90-day default was chosen without data on how long stakeholders refer back to shared roadmap snapshots. If teams share views with stakeholders who return to them months later, 90 days may be too short. Open question: make expiry configurable per workspace, or extend the default after collecting post-launch usage data.
Meridian Coalition Retrospective Design Document
Section titled “Meridian Coalition Retrospective Design Document”Status
Section titled “Status”Draft
Problem
Section titled “Problem”The Meridian initiative closed in March when the primary funder withdrew. Eleven coalition members gave sustained volunteer effort over eighteen months. No deployment happened. No continuation path exists. The coalition received a thanked-too-briefly closing message and has not heard from me since.
A retrospective document is owed. The constraints shaping what that document needs to do:
- Its audience is eleven people who spent real time on something that did not ship. They have a right to an honest account, not a managed one.
- I was the lead. The document cannot present me as a neutral observer.
- I got things wrong. Specifically: I kept the coalition aligned around optimism when some members needed to hear a harder truth earlier. I confused momentum with progress. A retrospective that omits or softens this fails its core function.
- I am writing this in a compromised state. The Meridian closure coincided with the loss of close contact with Celeste, a friendship of six years that drifted and then went. I do not fully know which loss I am grieving when I sit down to write. That is a live risk to the document’s reliability.
- There is no external deadline. The document does not exist yet because avoidance has no natural forcing function.
- The document, once sent, cannot be recalled. Eleven people will read it and draw their own conclusions about my leadership and about me.
The system being designed is the retrospective itself: its sections, its delivery protocol, and the sequencing of tasks that must precede it for the document to be honest.
Proposed Design
Section titled “Proposed Design”Document structure
The retrospective will consist of five sections:
-
What we built - A plain-language account of the technical and coalition work, without the growth-narrative framing I used in real time. Scope: what the infrastructure proposal covered, which coalition members contributed what, and what the state of the work was at closure. Target length: one to two pages.
-
What happened - A timeline of key decision points I controlled. This section names the February signals I saw and did not act on, and the March sequence that led to formal closure. It does not assign responsibility to the funder or to coalition members. It is an account of what I saw and what I did.
-
What I got wrong - Named directly, not softened into “lessons learned” framing. The decision to keep external communications optimistic past the point of honest optimism. The conflation of coalition morale management with transparency. The inadequate closing message. This section is the load-bearing one. If I cannot write it with specificity and without self-excuse, the rest of the document is a press release.
-
What the work was - A section that records what Meridian actually was, independent of whether it shipped. This exists to give coalition members a record of what they contributed that does not depend on a successful launch to have value. This is not a consolation prize. It is a factual record.
-
Closing - One to two paragraphs. No asks. No “let’s stay connected.” Acknowledgment that the inadequate closing message was inadequate, and that this document is the more honest version of what I should have sent then.
Delivery
The document will be sent by email directly to each of the eleven coalition members, not to a group list. It will not be published or shared beyond those eleven people without their consent. No response is requested or expected.
Sequencing with dependent tasks
The retrospective cannot be written before Theo’s message is answered. Theo was part of the coalition’s extended circle, and his April message touched on the closure. Writing the retrospective while that message sits unanswered creates an integrity problem in the document itself. Sequence: Theo message first, retrospective draft second, retrospective sent third.
Timeline
- Theo message: January
- Retrospective first draft: end of January
- External reader review: early February
- Retrospective sent to coalition: February
Alternatives Considered
Section titled “Alternatives Considered”Two-paragraph summary with a lessons-learned list
This is the version that would have been easiest to send in April. It does not fulfill the obligation. Coalition members gave eighteen months of volunteer time. A document that runs two paragraphs signals I have not taken their contribution seriously enough to give it a real account. Rejected.
A group video call instead of a written document
A call would allow dialogue and might feel more human. The problem is that I cannot predict or control what I would say under questioning before I have written through the honest account. A call before a document risks producing a managed narrative in real time. The document comes first. A call is optional after.
Not writing the retrospective
This is named here because it is the alternative I have been choosing by inaction since March. The consequence: I close the year having thanked eleven people inadequately for eighteen months of work. Named and rejected.
Risks and Open Questions
Section titled “Risks and Open Questions”Self-justification risk in “What I got wrong”: The natural pressure when writing about one’s own errors is toward softening, context-setting, and proportion-shifting. Mitigation: write that section first, before the surrounding narrative is established. Evaluate it against the question - would a coalition member who felt burned by the closure find this honest?
Audience readiness: Some coalition members may not want to receive this document. There is no clean way to pre-check without surfacing the retrospective before it exists. Risk accepted: the document is written as if they want the honest account, because that is what they are owed regardless.
Author state during drafting: I am not writing from a stable vantage point. The Meridian closure and the loss of contact with Celeste happened in the same year and I do not always know which one I am in when I sit down to write. Mitigation: write in short sessions; review each session’s output against the question - is this grief on the page, or is this a retrospective?
Open question - “What the work was” framing: I do not yet know whether that section reads as honest acknowledgment or as rationalization. It needs one outside reader willing to say if it crosses into consolation-prize framing before it is sent. That reader has not been identified yet.