Ran the same research task through three hosting setups and tracked cost per completed task. The expensive-looking option was the cheapest per outcome, because it failed less and never needed a retry.
| Setup | Cost/task | Failure rate |
|---|---|---|
| Budget option | $0.41 | 22% |
| Mid option | $0.38 | 9% |
| Premium option | $0.31 | 2% |
Lesson: measure cost per outcome, never cost per token.
The takeaway: Measure cost per outcome, never cost per token.