#StartupsEverywhere: Carl Peterson, Co-Founder, Thunder Compute
This profile is part of #StartupsEverywhere, an ongoing series highlighting startup leaders in ecosystems across the country. This interview has been edited for length, content, and clarity.
Boosting AI development by optimizing GPU access
Carl Peterson is developing a platform to tackle the shortage of graphics processing units (GPUs) in AI development. Thunder Compute rents out space on GPUs for cheap by using software to improve GPU utilization. We sat down with Carl to discuss his company, their experience in markets abroad, and the challenge of navigating varying state rules as a small startup.
Tell us about your background. What led you to Thunder Compute?
I met my co-founder at Georgia Tech and our time there is where the original idea for Thunder Compute came from. He was in a research lab doing systems for AI research where a common issue they encountered was lack of access to GPUs. The deadline for a paper would come up and then suddenly everyone would need the GPUs at the same time. There weren’t enough GPUs to meet demand, and even when they were available, they weren’t being used efficiently. A student would get their three-hour block scheduled a week in advance and end up only using the GPU for a fraction of the time because some line of buggy code prevented them from running the program until they fixed it. The scheduling blocked access to the GPU, but the GPU itself was sitting there idle and wasted.
After graduation, my co-founder and I went our separate ways. I continued to think about this issue and eventually got back in touch with him. We learned that it was on both our minds, especially with the AI boom and associated shortage of GPUs. So, we started experimenting with it and built some prototypes. Fast forward, we have a working prototype, I quit my job, Brian quit shortly afterwards, we got accepted into Y Combinator, and have been growing Thunder Compute to where it is now.
What is the work you all are doing at Thunder Compute?
Thunder Compute is primarily used by AI developers to get access to the compute they need to build their models and products at a fraction of the cost. There are two main components to Thunder Compute. First, we’re a public cloud platform for accessing GPUs. Second, is what actually powers that and allows us to be way cheaper than everyone else—a virtualization stack that allocates GPUs at a granular level and minimizes wasted processing space. We’ve built this from the ground up and it is our core competitive advantage.
Nvidia manufactures GPUs and sells most of them to Amazon Web Services and Google, who then rent them to developers. In many cases, when someone rents a GPU, it ends up sitting idle. For example, a client will rent 10,000 units from Google, and 90 percent of the time they’ll just be sitting idle. Then the GPUs are just sitting there, using some electricity, and not doing anything. This is the problem Thunder Compute solves. Infrastructure providers need a new software paradigm to free up the capacity. Right now, that doesn't exist.
Proposals have considered imposing know-your-customer and monitoring requirements for AI infrastructure providers. How would that impact the operations of Thunder Compute?
The burden of collecting that data would make it very hard to compete, the paperwork alone would be a burden. From a direct-to-consumer business model standpoint, if you are a self-service cloud platform, a huge part of the customer experience is having a low-friction setup process. Requiring us to collect IDs and Social Security numbers would drive away most of our customers and effectively push smaller providers like us out of service. Moreover, unless someone voluntarily tells us what they’re running on our GPUs, we have no way of knowing, and frankly, we don’t want to know. If we had to verify what everyone runs, we could prevent some bad actors, but also we would be compromising our user’s intellectual property and then incur a bunch of regulatory liability. I think it’s a non-starter. If that kind of monitoring were law, there would be no way for anyone to launch a new cloud platform without hundreds of employees, millions of dollars, and an entire regulatory office within their company.
What challenges have you faced navigating legal frameworks that vary across state lines?
There's a lot of confusion on working across state lines. Internet-based businesses hit state compliance grey areas quickly. When you operate a business in a state, you have to determine when to start collecting and filing taxes in that state and when to create a registered entity; now, multiply that process by fifty. If we had to do that on day one with zero revenue, it would be a huge headache. The more money a company has on day one to meet proposed strict compliance laws would make it proportionally easier, but at what point does that stop being a reasonable expectation and become a cap on innovation? We need a single, uniform, set of laws that encourage people with big ideas to pursue them, rather than punishing them for trying.
How have regulatory frameworks abroad shaped where you can serve customers? What lessons should U.S. lawmakers draw from that?
The European Union (EU) has very strict data regulations that often mean data can’t be transferred across borders. We have received a lot of interest from EU-based startups and researchers who would have used us because our platform is easier and more affordable than what they currently have access to, but can’t because we don’t have servers based in Europe. From our perspective, we lose the customer, which is disappointing, but from their perspective, they're forced to buy from a more expensive cloud provider and have a worse experience. I think the EU is a case study of what to avoid when thinking through policies and their consequences for startups.
What are your goals for Thunder Compute moving forward?
We want to expand our cloud offering as much as possible to test and prove our virtualization layer–eventually, we want that to be our main product. I want to make Thunder Compute do for GPUs what VMware did for central processing units (CPUs). Before VMware, CPUs had the same utilization issues, now they’re reaching 90 percent utilization. If we can scale this to the millions of GPUs cloud providers have, I think we can make a huge impact on access to compute, energy efficiency, and democratizing AI innovation.
All of the information in this profile was accurate at the date and time of publication.
Engine works to ensure that policymakers look for insight from the startup ecosystem when they are considering programs and legislation that affect entrepreneurs. Together, our voice is louder and more effective. Many of our lawmakers do not have first-hand experience with the country's thriving startup ecosystem, so it’s our job to amplify that perspective. To nominate a person, company, or organization to be featured in our #StartupsEverywhere series, email advocacy@engine.is.
