Fireworks says you’re wasting money on AI unless you’re using all the models
You are always better routing between a few models than to do one singular model." That's what Fireworks does through Nexus and its API, alongside competitors like Together AI, Baseten, and DeepInfra.
- Ferguson told The Register in an interview that while Fireworks AI, a managed inference provider, supports on-prem AI, "we're trying to push people more and more away from it because frankly, it's just the economics." The GPU hardware is just so expensive that if you can't keep it fully utilized, it's a waste of money, he contends.
- It actually matters when you're training that you go back to precisely the right machine, because if you don't, the other machine will still answer the question, but it'll answer in a slightly different way.
- "People are just trying to figure out more and more things to use AI for and [to answer questions like] just how much more can I send to cached models, how much more can I send to open models, when do I need this specialization?" he said.
Fireworks AI is fine with helping companies run AI models in their own datacenters, though VP Rob Ferguson argues against doing so.
For companies with particularly sensitive workloads, Ferguson said, Fireworks is happy to work something out – it offers a service called Sovereign for just that purpose and it has partners like Microsoft that can help.
- As a managed inference platform, Fireworks AI routes inference tasks to whichever model, frontier model subscription, or API is best suited to handle them.
- We have to be able to do all this where we're routing through a complex fabric through potentially more than one provider depending on what everything is.
- And so we're really systems experts." Hyperscalers, he argues, will give you a GPU but then it's your responsibility to put it to use.
- Fireworks' advantage involves being able to maximize utilization at scale while maintaining model precision and speed.
- He said Fireworks was working with a data science company and was caching about 90 percent of the tasks.
How this summary was made: extractive summary — built from the sources' own reporting. It is drawn only from the reporting of the sources listed below — read the originals for full detail. How we summarize →