Selective State Space Models: Solving the Cost-Quality Tradeoff
As AI is increasingly used in production scenarios, costs are mounting. Are alternative architectures the solution?
For now, most GenAI startups are focused on completing paperwork and are built on prompts. That may change in the months ahead.
It takes time for people to build useful things on new technologies.
The most famous example of this is mobile: The iPhone launched in 2007, but today’s most popular apps, like Instagram, Uber and WhatsApp, weren’t started till several years later. The same is likely in generative AI. While many people woke up to GenAI with ChatGPT in November 2022, it’s only now that we see a generation of new apps coming to market.
As we’ve spent time with founders building “AI for XYZ” businesses, we’ve noticed some common traits. These include:
Most AI solutions are focused on completing paperwork, not making decisions. These include generating SOAP notes in an electronic healthcare records platform (EHR) for doctors to review, case files and briefs for lawyers, and even daily incident reports for police officers. The specifics of the paperwork depend on context, but the idea is the same: free people from filling out forms so they can spend more time on their job.
Nothing has been fully rolled out yet. Everyone is running experiments. Even in relatively mature areas, like AI-powered customer support, companies are not wedded to their solutions. People are keen to experiment and are mindful that everything may look different within a year.
The generative capabilities of most apps are built on prompt tuning large foundation models like GPT-4. That may change in the next few months. Llama 3, announced in April, is a significant step forward for open source models, with Llama 3 70B approaching GPT-4 levels of performance and Llama 3 400B (presently in training) promising even more.
It still feels very early in the evolution of AI-powered apps and we’re encouraged by the quality of teams that are inspired to build. Some of these are domain experts with a deep understanding of business processes. Others are AI-natives with an intuition for the technology and how it’s likely to evolve.
We work with both at BCV Labs, through our fellowship and incubation programs. If you have ideas for how AI can be applied to specific business problems, we’d love to hear from you.
As AI is increasingly used in production scenarios, costs are mounting. Are alternative architectures the solution?
Cube is the standard for providing semantic consistency to LLMs, and we are investing in a new $25M financing after leading the seed round in 2020.
In this edition of “In the Lab,” Amit Aggarwal explains why he’s building an AI startup in BCV Labs after selling his company The Yes to Pinterest.