The pilot was pitched as free. The vendor ate the inference cost for the first six months and the IT team marked the project as zero-budget. By month four, the real spend was approximately $180,000, none of which sat in the line item anyone was watching.
Where the money actually went
Three buckets. First, the eval set — roughly 400 labeled conversations a domain expert had to produce and maintain, at the fully-loaded cost of that expert's time. Second, the human-review queue for the 6% of outputs the model flagged as low-confidence, which ended up being a half-FTE no one had hired. Third, the integration debt: three engineers, two months, to wire the copilot into the four systems it actually needed to read from.
Inference, the line item everyone watches, was less than 8% of the true cost. The vendor's promotional pricing was real. It was also irrelevant.
"Inference was 8% of the true cost. The vendor's free pilot was real — and almost completely beside the point."
The team that ran this rollout now budgets every AI pilot with three line items: model spend, eval maintenance, and review-queue staffing. The model line is usually the smallest. The eval line is usually the one that gets cut first and then quietly destroys the rollout in month seven.
If you are evaluating an internal copilot right now, the honest question is not what the API costs. It is who, on Monday, owns the eval set. If you do not have an answer, you do not yet have a project.
Dispatch Newsletter
One operator-grade breakdown every Tuesday. SEA tech, AI economics, execution frameworks. No fluff.
