Since OpenAI released ChatGPT late last year, terms like generative AI and large language models have been on everyone’s lips. But when you dig beneath the hype, large language models generally require very expensive GPU chips. SambaNova is introducing a chip today that will significantly reduce that cost while handling 5 trillion parameter models.
SambaNova may not be a household name like Google, Microsoft or Amazon, but it has been building a full-stack AI solution that includes hardware and software for years, and has raised more than $1 billion per Crunchbase from investors like Intel Capital. BlackRock and SoftBank Vision Fund. Today, the company unveiled its latest chip, the SN40L, which is the fourth generation of its in-house custom AI chips.
Company founder and CEO Rodrigo Liang says the idea behind creating his own chips is to control the underlying hardware for maximum performance, something the world increasingly needs to process large language models that are resource-intensive.
“We need to stop using this brute force approach of using more and more chips for large language model use cases. So we went out and built the SN40L which is specifically tuned for very large language models to enable AI for enterprises,” Liang told TechCrunch.
“How many resources does it take to actually run a trillion printer model like the GPT-4? I can do it in eight sockets, I can deliver it on prem, and I can deliver it fully optimized on hardware and you get state-of-the-art accuracy,” he said.
It’s a bold claim, but Liang says its new chips are 30 times more efficient by reducing the number of chips needed to power these models, and because the chips are built for SambaNova software, they’re configured to run at maximum efficiency for the software. In fact, he claims that it would take 50-200 chips to run the same trillion parameter model on competing chips, while Sambanova has reduced that to just 8 chips.
SambaNova delivers a full stack hardware and software solution for building AI applications. “We’re in the business of building AI assets, which allow you to quickly train models based on your private data, and that becomes an asset to the company,” he said.
He points out that although SambaNova helps customers train models, it is under their ownership. “So what we tell the customer, is your data and your model. After we train a model on your data, we transfer ownership of the model to the company forever.”
By providing the hardware and software solution as a multi-year subscription, Liang says customers are assured of more value on their AI projects.
The new SN40L chip is available starting today, but is fully backwards compatible with previous generation chips, according to the company.