Selective State Space Models: Solving the Cost-Quality Tradeoff
As AI is increasingly used in production scenarios, costs are mounting. Are alternative architectures the solution?
BCV is leading the $20 million seed round for Contextual AI, which will develop a platform for enterprise LLMs.
Hateful memes. That’s how Douwe Kiela and Amanpreet Singh, the co-founders of Contextual AI, first met. At Facebook AI Research, their job was to find programmatic ways of combating these misery-producers, which is more difficult than you might think. It was the first in a series of difficult problems they took on together at FAIR and Hugging Face, all technically challenging and with real-world consequences.
We first met them early this year and were instantly drawn to their rare combination of pragmatism and intellectual depth. Our conversations started with their seminal work into retrieval-augmented-generation for large language models (LLMs). We had previously noticed their research addressing issues like hallucination when we found it on the desks of CTOs building on LLMs.These early discussions broadened to cover the range of business problems that generative artificial intelligence could address. Many corporate buyers had shared with us the conflicting emotions evoked by ChatGPT: wonder, at how it could automate work well beyond what was previously thought possible, and fear, about data privacy and the inaccuracy of information emanating from these models. It was immediately clear to us that Douwe and Aman’s ideas, which they were going to use as the basis of a new business, had direct relevance to many of the biggest challenges faced by enterprises adopting generative AI. Within days, we’d agreed to partner and lead their company’s seed round.
Today, we are delighted to announce our partnership with Contextual AI. BCV is leading its seed round of $20 million, with participation from our friends at Lightspeed, Greycroft and a host of notable angel investors including Elad Gil, Lip-Bu Tan, Sarah Guo, Amjad Massad, Harry Stebbings, Fraser Kelton, Sarah Niyogi, Timothy Li, Chris Potts, Kyunghyun Cho, Orhan Firat, Nathan Benaich, Joe Spisak, Edward Greffenstette, Karl Moritz, Anton Troynikov, Nathan Benaich and SVA.
The company is off to a fast start. Douwe and Aman are working from BCV Labs, our new AI incubator in Palo Alto, so we are often shoulder-to-shoulder. They have already proven to have the two key skills most needed by any AI founder. One is the ability to recruit in a competitive market — in a very short time period, Contextual has grown to a dozen people, all outstanding technical talents with commercial and research experience in AI. The other is the ability to secure graphics processing units (GPUs), required for AI infrastructure, on favorable terms, which nowadays calls for more ingenuity than you might expect.
We are at the early stages of understanding the full potential of generative AI, with the landscape changing on an almost weekly basis. Enterprises realize its transformative potential, but are still thinking through how best to leverage it in the service of customers and to make internal operations more efficient. Contextual’s enterprise solutions will be grounded in the Retrieval Augmented Generation (RAG) method, which Kiela helped pioneer while at Meta.
Our view is that AI will become ingrained in everything enterprises do in ways that are secure, deeply integrated and, most importantly, contextual to employee workflows. Spending time with Douwe and Aman, it’s clear they both understand that at an intuitive level. We are thrilled to be working with them both to bring the power of language models into the workplace.
As AI is increasingly used in production scenarios, costs are mounting. Are alternative architectures the solution?
Cube is the standard for providing semantic consistency to LLMs, and we are investing in a new $25M financing after leading the seed round in 2020.
In this edition of “In the Lab,” Amit Aggarwal explains why he’s building an AI startup in BCV Labs after selling his company The Yes to Pinterest.