A chat feature at tens of millions of DAU

Context

A consumer app ships an AI chat feature to tens of millions of daily users. Volume is enormous and compounding, most traffic runs on a cheap model, and the team can keep GPUs busy.

The decision

At this scale the crossover to self-host arrives within the first year — pay-per-token can't compete once utilization is high and volume compounds.

What most miss

The API bill looks fine in the pilot and becomes a scary line item by month nine. The cliff is a function of growth × utilization, not today's invoice.

Stakes

Staying on usage-based pricing past the cliff at consumer scale is a seven-figure annual overspend.