Validation Is Not Optional: The Three-Phase Engagement
Scope, validate, deliver. The middle phase is the one everyone wants to skip, and the one that decides whether the engagement is worth attempting at all.
The most useful frame for the day-to-day rhythm of an FDE engagement comes from Colin Jarvis, who runs OpenAI’s forward-deployed engineering team. It has three phases: early scoping, validation, and delivery. Independent operators across the industry have converged on the same three cuts under different names, which is itself a signal that the structure is doing real work.
The frame’s key move is making explicit what less mature engagements blur: validation is not a sub-phase of delivery. It has its own deliverables and its own decision gate. Most failed AI deployments treat validation as something that happens after delivery: a post-launch review nobody acts on. In a working FDE engagement, validation decides whether delivery is worth attempting at all.
Phase one: scoping
Scoping lasts a few days. The FDE sits with users, watches the work, maps the processes, and walks away with a prioritized list of things that could be built and an opinion about which one is most likely to produce a measurable outcome.
The single most important question: is the problem one of the customer’s top five priorities? If not, the engagement should not proceed. This has nothing to do with technical feasibility and everything to do with organizational physics. Production AI deployment requires sustained senior-leadership attention over months. If the project isn’t on the top-five list, legal deprioritizes the review, IT applies the standard six-week request cycle, and the line workers treat the rollout as one more initiative to wait out.
Scoping’s second job is verifying that the stated problem is the real problem; sales conversations almost always produce a stated problem one level removed from the truth. The third is identifying the sponsor, the senior person with the political capital to grant exceptions to standard IT, security, and procurement processes. If you walk out of scoping without a named sponsor, you don’t have an engagement. You have a sales conversation that’s been mislabeled.
Phase two: validation, the gate
In standard consulting, the next step after scoping is a statement of work. In FDE work, the next step is to ask whether the scoped thing is actually worth building. Build the evaluation criteria, run the proposed system against a representative slice of real data, and present the results with a recommendation. If the results clear the bar the customer cares about, proceed to delivery. If not, adjust scope or end the engagement honestly.
A story shows what the gate is worth. A vertical AI startup was deploying a store-associate assistant for a four-hundred-store retailer. The scoping document set a validation threshold of seventy percent on a hundred-case eval set. The temptation inside the team was to skip validation and go straight to a pilot in one store; the customer’s own product manager was pushing for exactly that. The FDE lead held the line, and the baseline run came in at forty-one percent.
The failures weren’t the ones anyone expected. The system couldn’t distinguish loyalty-program members from walk-ins, which determined the right answer to return questions, and it couldn’t handle clearance markdowns governed by a rule set that changed every Friday. Both gaps were obvious in retrospect, and neither had been visible in scoping. Two weeks of fixes took the retest to seventy-three percent. The pilot launched in twelve stores and reached forty-eight by week ten. Without the gate, the team would have put a forty-one-percent system in front of real customers, and the damage to the customer’s confidence would have been hard to repair.
Validation looks expensive: one to three weeks of senior engineering time that produces no production code. That accounting is wrong. The cost of skipping it is not those weeks. It’s the four months of delivery work done against the wrong target that then has to be redone.
Phase three: delivery
Delivery is where the work looks most like classical software engineering, with a few characteristic differences: instrument the system from the first commit, keep the customer’s dashboard live and visible from day one of production, and design for handoff from the first week. The FDE is going to leave, and the system has to keep running. The runbook, the alerts, the eval set, and the playbook for the customer’s internal team are part of the delivery, not afterthoughts.
The frame doubles as a diagnostic for engagements in trouble. If the customer can’t articulate the top-five priority their project addresses, you’re stuck in scoping. If the eval set exists but the customer hasn’t signed off on the threshold, you’re stuck in validation. If the system is in production but the metric isn’t moving, you’re stuck in delivery. Naming where you are is the first step in deciding what to do next.
Want the full playbook, with the templates, the interview transcripts, and the week-one checklist? It’s all in my book Forward Deployed AI Engineering: A Working Guide to the Hottest Job in Software, available on Amazon: [Amazon link]