The Token Price Is a Decoy, in a Bite

Fireworks AI wants operators to stop pricing AI work by the token. Vice president Rob Ferguson told The Register that cost per task is the number that decides a budget, and almost nobody tracks it.

• Fireworks claims routing each request across a portfolio of models makes the bill six to eight times cheaper, and says it can guarantee at least half.
• The company says one data science customer cached about 90 percent of its tasks, against 30 percent for Claude Opus in a comparison Fireworks ran and has not published.
• At a cache read priced at one tenth of the input rate, going from 30 percent hits to 90 percent cuts the effective cost of input by about 74 percent before anything is routed.
• Ten thousand GPUs is roughly a billion dollars of capital, or $2.85 per GPU-hour across four years, paid whether or not the silicon is busy.

The honest read is that caching carries most of the claimed saving, and cache hit rate belongs to the workload rather than the vendor. Read the full analysis.