Selective State Space Models: Solving the Cost-Quality Tradeoff
As AI is increasingly used in production scenarios, costs are mounting. Are alternative architectures the solution?
In this edition of “In the Lab,” Amit Aggarwal explains why he’s building an AI startup in BCV Labs after selling his company The Yes to Pinterest.
“In the Lab” is BCV Labs’ interview series, where we sit down with a member of our Labs technical community, whether they are currently working in our incubation space or have set off on their own. This edition’s guest is Amit Aggarwal, who you can connect with on LinkedIn. And shoot us a line at bcvlabs@baincapital.com if you’d like to learn more about what we’re building.
Just shy of two years ago, Amit Aggarwal and his co-founder, former Stitch Fix exec Julie Bornstein, announced they were selling their company, The Yes, to Pinterest to accelerate Pinterest’s e-commerce strategy. It was a high-point in his over 20-year-career, and no one would have faulted him for wanting to ease up from the grind. But today, we’re happy that Amit is in BCV Labs’ incubation space, working on a new AI startup in stealth.
For Amit, starting a new company again was an obvious decision: He’s confident that he can. He’s felt the pain his new company is trying to solve before and the problem is felt across every business segment and sector. He believes that his new company will be able to produce a product 5x better than its competitors, which you’ll see is advice he recommends to other technical founders, as well.
Amit grew up in India and, briefly, Libya. “One of the themes of my childhood is change,” he said, noting that he and his family moved every two or three years. “And I think that was actually really helpful, because it helped me learn how to adjust to new things very, very quickly.” He had dabbled in programming while a high school student in the early ‘90s, but it was an engineer friend who convinced him that computers would be the future and that he should study computer science at the Indian Institute of Technology, Delhi. Turns out it clicked with him immediately and he never looked back.
He moved to the United States to continue his studies at the University of Washington and headed to Silicon Valley in 1999, where he embarked on a career that’s included stints on pioneering search teams at Inktomi, Google and Microsoft, a senior engineering role at Groupon, and over four years as CTO at BCV portfolio company Bloomreach, where he developed a relationship with our team that led to six months as our entrepreneur in residence and an intro to Julie, who would be his co-founder for The Yes.
Amit is the type of founder who is as thoughtful about product as he is about the art of leadership, and he took the time to share his hard-won lessons from both ruminating on the past 25 years and being in the trenches now.
The following transcript is edited for length and clarity.
At what point in your career did you know that you wanted to start a company? When did that become a real thing for you?
It was the early days of Google. And Google is a great company — I loved it. But I remember asking myself a very simple question: Will Google be fine without me? In your life you sometimes have these moments where you ask something that ends up super clarifying to you. The answer for this was very clearly, “Yes, Google will be fine.” And it was enlightening because the answer was not satisfying for me.
I wanted to be at a place where I felt like I was needed, where I was having a real impact and the success of the company depended on me doing a great job.
I left Google and I actually didn’t have the stomach to start a company at that point. I knew I wanted to, but instead I went to a company that was smaller, and from then on I kept going smaller and smaller.
Smaller and smaller startups until you just made your own.
Yeah, exactly.
What made you take a bet on BloomReach after Groupon? It was still a young company when you joined.
I don’t think it was a logical decision! It was an emotional one. Someone once told me that as you become more senior and mature and grow in your career, the big decisions become more emotional than logical.
When you’re earlier in your career, you can break down opportunities in a spreadsheet across compensation, growth, and learning. You give a score to each one, and then you just pick the one with the highest score. But one sign of seniority is you know what you want. The reason to go to Bloomreach was to learn how to become an engineering leader, and that was a huge difference from being a manager at Google and Groupon. The difference was not so much in the size of the team but rather was about the gap between leading a big team inside Google versus being the head of engineering at a startup, and the learning that comes with the latter is just at a different level.
I also really respected the people at the company. I thought the technology was really good. So when the opportunity came, it was clear to me that it was something that I would be really excited about doing.
Bloomreach has been through a few phases. Which phase did you join in and what phase did you leave in?
The company was about 50 people when I joined, and it was shortly after raising its Series B. When I left there were around 150 or so people. I saw a lot of ups and downs during those five years I was there.
The company now has around 1,000 people and is doing really great, but I was there when the company was investing in a new product, which was a time of transition. It was neither the phase of pure growth nor the early phase, it was the phase where the company was trying to pivot. It was an amazing time for learning.
What did you learn from Bloomreach that informed how you built The Yes?
Before Bloomreach, I’d never been a senior executive at a startup of that size. So most of my learning there was around how do you build your initial product, how do you get to product-market fit, how do you do enterprise go-to-market, how do you iterate on the product?
I also learned that there’s a lot to running a business that you don’t get at a big company. When you’re at a big company, you have your team and you have your goals, and you’re not concerned about the financials of the company, you’re not concerned about how the company is doing. At a startup, you’re thinking about everything. You’re thinking about how the financials of the company impact the product. You’re thinking about how much setup is required in the product and how that impacts the go-to-market.
But I think that the most important thing I learned at Bloomreach was that all of that is fun. And I’ve seen people go two routes after their initial startup experience. Some people say, I love it, I’m going to do more of it. And then some people say, this is not for me, I’m going to go back to a big company. The Bloomreach experience showed me I that I loved the startup journey. Why would I do anything else?
You learned by observing, but when you actually went over into that leadership position yourself, how were you able to figure out how to balance all of that and keep eyes on everything in a way that worked, and what needed micromanagement vs. big-picture thinking?
I had good mentors at BloomReach. Raj and Ashutosh, the founders, are great people to work with, super supportive. I learned a lot by observing the exec team.
One of the things I learned about startups is that there is no one right answer. The beauty about a startup is that, when it comes to micromanaging or not, the answer really depends on the current point of time. The traditional wisdom on how to do things is thrown out of the window.
So just to give you an example, there was a time when we were in a competitive situation where an early customer was evaluating another solution, and it was super critical for the success of the product to win that customer. I got so into the weeds, to the extent that I became the point of contact for the customer, and that was when we had a team of around 50, so it wasn’t very small. But I was still doing that and delegation in that case was not the right answer.
There are also situations where you want to be hands-off because you want to focus on the bigger picture things. When I joined Bloomreach, a number of the senior engineers had been there for a few years and were asking questions about their career prospects. And so at that point, the right thing for me to do was to answer them, and decide how we were going to think about careers at Bloomreach.
So one scenario required me to go into the nitty gritty of making a product successful and the other required me to step back and work on career ladders for the engineering team. There was never a consistent right answer for which path to take. That to me is where the fun of being at a startup comes in.
It sounds like even though you were CTO, you had a lot of insight and exposure to all these things that maybe aren’t necessarily what people consider to be part of an engineer’s role. Do you recommend that early-stage CTOs get to that level of involvement in the business?
They should absolutely be involved. I actually would go even further and say that’s a necessity.
I have a different view on engineering teams, as well. I don’t like to hire engineers who say they don’t think about the product. If someone says, “I’m going to build what you tell me to build,” I won’t hire them. Big companies are different, but a startup is a combination of technology, product and go-to-market. As an engineer, you cannot build a product without having a good understanding of the trade-offs involved in something like whether a product should be self-serve, for example. If you don’t understand the go-to-market implications and you don’t understand the fact that adding a certain config is going to add lots of work for the customer and the customer success teams, then you’re not doing your job.
And especially as a CTO, you have to understand them. So I actually don’t see how a CTO can’t think about go-to-market and product.
How did you eventually decide you wanted to spend time on The Yes?
Before starting The Yes, I was an EIR at BCV, and so I want to thank the Bain Capital team for supporting me during that six-month process, which in some ways was a process of discovery.
When you’re starting a company, you’re learning about yourself, and I went through multiple phases. The first phase I went through was as a generalist engineer, where I felt like I could pretty much do anything: I could build a security company, I could build a supply chain company, etc. You could call it arrogance, but I quickly realized that it was not a good idea. You need to have an intuitive feel for the problem that you’re solving as a startup founder — you need to feel that you know more about this problem than anyone else.
I realized there are three main ways of starting a company.
The first way is you really feel deeply passionate about a problem and you deeply understand it. You have an intuition on how to solve that problem way better than anyone else. That’s a great way to start one.
Another way is you do a market survey, see what’s going on in an industry, map it out and then find an opportunity. That’s how a management consultant would approach things, but it’s viable.
And the third way, which is what happened with The Yes, is you partner with someone who has a deep intuition about a problem that you also feel passionate about and know you can add a lot of value to their solution. Julie was thinking about personalization in e-commerce, and personalization was a problem I had worked on at Bloomreach, and so BCV connected us. I felt like personalization still wasn’t solved, and that there was a real opportunity for us to try a different approach.
What did you look for in Julie to determine that this is the person you wanted to spend the next few years building something with?
First I had to consider the opportunity itself. Number one was the market. Was it a big market? Was it ready for disruption or were incumbents already doing it well? Then I asked myself, is this a challenging technology-product problem? That’s not a general requirement to build a company, but it’s a requirement for me personally, as a technology-product person. If the core innovation is outside of that, that’s not the right startup for me.
And then with Julie, as with any co-founder, was considering where our values matched. Would I be able to work with this person in the next 10 years? Do we have complementary skills? Her background was in retail, which would complement my background in technology, but then I needed to evaluate if she really believed in technology. That’s important to me.
But look, at the end of the day, we spent about six months talking to each other, it’s why people usually call it “dating,” because there’s no substitute for talking with a potential co-founder and just forming a gut feel of whether it would work or not.
Many people with the experiences you’ve had would hang up their boots and go angel invest and advise. Why incubate again?
Well, I would be really bad at angel investing! But I think that the serious answer is I don’t think of this as pain. Really. I’m loving it.
When you start a company, you want to build a big business. The Yes was very successful and we’re very fortunate to have had an acquisition — but it was not a big business, it was not a billion-dollar company. And then, to get back to my three ways of starting a company, I felt like I had an idea that I felt deeply passionate about. This new one is addressing a problem that I feel deeply connected to because I’ve experienced it at other places.
After the acquisition, I didn’t need to go do a survey or figure out which of five ideas I had would be best. I knew exactly what I wanted to do. I feel like that’s a rare thing, and when that happens, you have to take it.
If you get an idea where every time you talk about it with people or look at other companies in the space and your conviction is only going up, you have to pursue it. I also felt that I’d learned a lot of lessons from The Yes and I wanted to apply them.
Starting a company can be a painful journey if you think of it as pain, but like I’ve said, I don’t!
You have to love the process is what I’m hearing.
You have to love the process. You have to love the journey.
It seems that every market in AI is hypercompetitive. How much do you think about competition? How does it factor into your decisions? What advice would you have for people in competitive markets?
I don’t think about competition a lot. At the same time, you want to be aware of what is happening around you. This is an one example of the many contradictory beliefs that you have to juggle at a startup. The thing about competitive markets is that by nature there is always going to be competition. Maybe I just have a simplistic view on this, but I think that often when companies fail, they’ll blame their failure on a competitive market. The truth is that their product just wasn’t good enough. If you’re in a competitive market, you can’t build a 10% better product and hope to succeed. You have to build a disruptively better product.
I wouldn’t have started this new company if I didn’t believe that we could build a 5x better product. Now, doing that is super hard. But if you lose, it will not be because you had too much competition, it will be because you didn’t actually build a 5x better product. My worry is not if someone is going to do it before us.
What expectations do you have of your early investors?
My main question is how long-term they are. Are they long-term thinkers who have gone through the process of building companies, which inevitably comes with ups and downs? One of the things I learned from Bloomreach’s CEO, Raj, is the power of persistence, and I think early investors can help with long-term thinking and support when things are hard.
At Labs, I routinely hear about the great culture you’re building now. What does culture mean to you, and are you creating it intentionally?
I read something recently that said values are only interesting to the extent that the opposite of your value is also viable. For example, a company is not going to say, “Being honest” is one of their values, because honesty is table stakes. It’s not viable to ever have the value of being dishonest. And so “being honest” isn’t a great value. .
Culture, then, is the core of who you are. We picked three values.
The first one is, “Build and invent.” When we see a big problem, we are not bogged down by it. We invent solutions and we learn. And the way we learn is by building. We chose that because in this competitive world of AI, that’s the only way to build the future. We’re in an amazing time. The companies that will succeed are the ones that are going to innovate.
Value number two is, “Play to win.” We’re all at the company for a reason.
And the final one is, “Be comfortably candid.” We’re deliberate about calling out bad behavior and rewarding good behavior, to stay focused on inventing and building the company.
We often meet engineers through BCV Labs that want to start companies. Do you have any advice for them and the experiences that they should seek to have before they start the company?
Experience at a startup obviously helps If you want to start a company. Being at Bloomreach gave me the training and the confidence to start a company. That said, I don’t think it’s necessary. Just keep finding ways to continue learning and growing.
My main advice to engineers wanting to start a company is what I said earlier. You need to have a deep resonance with the problem you’re considering, and you need to be confident you can build a 5x better solution than competitors.
What kinds of people should join you on your mission with your new company?
They need to resonate with the values I mentioned, and really want to enter this AI market and be motivated by imagining a few years from now where we’ve built a game-changing product that they’re proud of.
We’ve taken a different approach to building this new company from my other experiences, in that we have not hired outside the Bay Area and have kept the team really small. There’s a difference between invention-oriented products and execution-oriented products. Execution-oriented products are ones where you know what to do — it can be difficult, but, for example, if you’re building an e-commerce site, there’s a known path. Remote work and dispersed teams can work for products like that. Invention-oriented products are what a lot of today’s AI products fall into, including our own, and that involves a lot of trial and error. Those products are not conducive for remote or big teams. Invention doesn’t work well with huge teams that are separated. We’re in that bucket and are looking for a small team.
We have a very high bar for technical talent and we want people who want to build and invent.
As AI is increasingly used in production scenarios, costs are mounting. Are alternative architectures the solution?
Cube is the standard for providing semantic consistency to LLMs, and we are investing in a new $25M financing after leading the seed round in 2020.
For now, most GenAI startups are focused on completing paperwork and are built on prompts. That may change in the months ahead.