Risks, Assumptions and Phased Rollout

What happens on the day the system is wrong

What happens on the day your system is wrong? A plan that can't answer is a plan for a demo. Behind any deployment sit assumptions that feel too obvious to write down: the data means what you think, you'll get access, people will use the thing, the output is right often enough. Some of those are wrong, and you rarely know in advance which ones.

The FDE's job is to find out cheaply and early, and to roll out in steps that let a wrong assumption surface before it reaches a customer. Two tools do most of that work. The first is a ranked list of assumptions, each with an owner and a cheap test. The second is a phased rollout, from shadow mode to assisted mode to automation, with a go or no-go check between each phase and a way back from every one.

A water utility's exceptions queue

A fictional regional water utility holds every anomalous meter read in an exceptions queue before bills go out. Clerks review about 1,800 exceptions a week, at roughly 4 minutes each, and decide whether to release the bill on an estimated read or send a technician for a field read. The billing manager wants a model to clear the obvious ones. Every number below is made up for the example.

The rollout moves one phase at a time: a phase that misses its go criterion stays where it is, a wrong bill traced in assisted mode sends it back to shadow, and a rollback from automation returns it to assisted mode.

The big-bang plan and its three hidden bets

The direct answer goes like this: "Train a classifier on last year's clerk decisions, deploy it to auto-release anything it's confident about, and free up the clerks."

It makes three bets without saying so. If last year's clerk decisions were inconsistent, the model learned the inconsistency. If the billing system can't accept an automated release, the launch stalls on an IT ticket. And if a wrong estimated bill goes out to 200 households, there is no switch to stop the next batch. This is a big-bang rollout: one jump from nothing to fully automated, with nowhere to stand if any bet fails.

Ranking what the plan assumes

The better move is to write the bets down and rank them by how badly the plan fails if each is wrong, then attach a test that costs a day or less.

The rows cover the usual risk families: data quality in row 1, access and approvals in row 2, change management and trust in row 3. Rows 1 and 2 are owned by the customer's people, so they go on the customer's calendar in week one.

Say the day-one re-review comes back with the senior clerk disagreeing on 9 of 100 past decisions, mostly where a read looked low because of a possible leak. Does that kill the plan? No, it narrows it. Leak-pattern reads stay manual in every phase, and the model only ever proposes on the remainder. This is the payoff of testing the top-ranked assumption first: a finding that would have poisoned a big-bang launch becomes a scoping decision on day one.

Shadow, assisted, automated

Shadow mode runs for two weeks. The model scores every exception while clerks see nothing and work as usual. In week one it marks 900 of 1,800 as high-confidence "release". Clerks released 882 of those 900, so agreement on that slice is 882 / 900 = 98%, and all 18 disagreements get reviewed by hand. The go criterion, agreed with the billing manager before shadow starts, is at least 97% agreement on the high-confidence slice in both weeks, with every disagreement explained.

Assisted mode runs for three weeks. Clerks see the suggestion and its reason, and accept with one click or override with a reason code. You measure accept rate, override reasons, minutes per exception, and any wrong bill traced to an accepted suggestion. The go criterion is an accept rate on the high-confidence slice at or above 95%, and zero wrong bills from accepted suggestions.

Automation covers the high-confidence slice only. Those exceptions release without a clerk, and a clerk audits a random 5% sample each day. The kill switch is a config flag that routes everything back to the clerk queue, and the billing supervisor can flip it without an engineer. Roll back if the audited error rate exceeds 1% in any week or complaints about estimated bills rise. If the slice holds at 900 a week, that is 900 × 4 = 3,600 minutes, or 60 clerk hours a week, without ever automating the hard half.

Rules for ranking risks and phasing a rollout

To build the list, walk each step of the plan and ask, "What has to be true for this to work?" Then rank by consequence, since likelihood alone misleads: a wrong assumption that ends the plan outranks a likely one that only dents it. Give every risk an owner and a test that costs less than a day. A risk with no owner stays on the slide, while a risk with a test either dies or turns into a plan change in week one, the cheapest week to change plans.

Phase by blast radius. Shadow mode lets the system be wrong at zero cost, on live data. Assisted mode lets a human catch errors while you measure trust. Automation comes last, on the narrowest slice that pays, with a sampled audit behind it.

Write the go or no-go criteria before each phase starts, with the sponsor. Why before? Criteria written after you see the numbers will drift to match them. Every phase also needs a way back that a non-engineer can trigger. If rollback needs a deploy, it will be slow on the day it matters.

Is shadow mode always worth the two weeks? Not always. Skip it only when a wrong output is cheap and instantly reversible, like a draft an agent edits before sending. Anything that reaches a customer or a ledger gets a shadow phase.

Where risk plans turn into slideware

The first failure is listing risks with no mitigation or owner. "Data quality might be an issue" is true of every project and changes nothing. The interviewer's follow-up is "so what do you do on Monday?", and the answer has to be a named test and a person.

Then there is the big-bang rollout from the example: one switch from nothing to fully automated, with no comparison period and no rollback. When it goes wrong there is no data on whether the model or the process was at fault, and no quick way to stop. A milder version is phases with no criteria. "We'll run shadow for a while and see" ends when someone gets impatient, which is a schedule rather than a test.

The last failure is ranking by likelihood alone. A probable but minor risk (volume varies) gets attention while an unlikely but fatal one (labels are noise) goes untested.

Drill: carrier invoices

Try this fictional brief. A mid-sized logistics company wants an LLM to read inbound carrier invoices (PDFs) and auto-approve the ones that match the agreed rate, about 5,000 a month, where accounts payable reviews every one today. In 15 minutes, write four assumptions ranked by consequence, each with an owner and a test that costs a day or less. Then write the three phases with a go or no-go criterion for each, and say who can flip the kill switch.

A strong answer puts something like "the rate card in the contract system is current" at rank one, because a stale rate card makes every auto-approval wrong in the same direction. Its automation phase covers only invoices under a value threshold that match exactly, and the rollback is a flag finance can flip. Rehearse it aloud with the FDE AI coach in thita, or run it as an FDE mock round and let the interviewer play the finance controller who wants it all automated by month end.

How this shows up in the interview

In decomposition rounds, risk shows up as the interviewer's injected constraints, which Aced's Palantir guide reports as part of the format. "The data owner won't sign off" is an access risk arriving mid-round. Treat it like a row in your table: who owns it, what the cheap test is, and which phase it blocks. Production readiness is scored in build rounds too. Sierra's own blog post on its interview describes a Review phase that covers production readiness alongside the demo and data models. Behavioural rounds probe the same instinct from the other side; one prep guide by Sundeep Teki lists a question about walking through a 2 a.m. production deployment failure and your ownership of it.

Expect the push: "The customer wants it fully automated next month. What do you say?" Credit goes to agreeing with the goal while keeping the phases, with criteria set in advance and a rollback someone on the customer's side controls. A sentence that holds up: "I'd run two weeks in shadow first, with a 97% agreement bar we set today, so that if we automate next month it's on the slice we've already watched work, with a switch your team can flip."

Check yourself

• Why does "past clerk decisions are consistent" outrank "volume is stable"? Answer: If labels are inconsistent, both the model and its evaluation are meaningless and the plan ends. Unstable volume only changes the savings estimate.
• In the shadow week, what was agreement on the high-confidence slice, and what had to happen to the disagreements? Answer: 882 / 900 = 98%. All 18 disagreements had to be reviewed and explained before going to assisted mode.
• What makes a rollback plan real? Answer: A trigger agreed in advance (such as audited error rate above 1% in a week) and a switch a non-engineer can flip without a deploy.
• When is it reasonable to skip shadow mode? Answer: When a wrong output is cheap and instantly reversible, like a draft a person edits before it goes anywhere.