Scott Stephenson led a group of physicists at the University of Michigan on a journey in 2015 that would change the way AI works. What started as a research project to learn more about dark matter turned into Deepgram, a cutting-edge voice AI company that is changing the way businesses use audio data. This is the story of how Deepgram started, the problems it faced, and how it became a leader in speech recognition technology.
The Spark: From Physics to Voice Data
Scott Stephenson was a Ph.D. student who was studying how to find dark matter in a bunker two miles underground. He was used to dealing with hard problems. He used end-to-end deep learning to “listen” for signs of dark matter interactions in his work. Scott, along with co-founders Adam Sypniewski and Noah Shutty, spent their free time being interested in “life-streaming.” They made wearable recording devices that could record hundreds of hours of voice notes, conversations, and interactions with other people. But they quickly ran into a problem: there wasn’t an easy way to go through all of this audio data and find useful information.
Scott saw a chance to use his deep learning skills from particle physics to audio analysis because he saw a gap in the market. Back then, traditional speech recognition systems used hand-made rules and small datasets, which made them less accurate and less able to grow. Scott and his team came up with a way to get around these problems by using end-to-end deep learning to build a strong, scalable platform for speech recognition. So, Deepgram was created with the goal of making sure that every voice is heard and understood.
Early Problems: Starting from Scratch
Many startups have trouble in their early days, and Deepgram was no different. In a market dominated by tech giants like Google and Amazon, the team had the tough job of getting investors to support their deep learning approach. People thought speech recognition was a crowded field, and they were worried that it would become a commodity. Getting money was hard, but the founders kept going and it worked out. Deepgram had raised $13.9 million by 2020. In 2022, they raised a huge $72 million Series B round led by Madrona and backed by investors like Alkeon. This money helped them with their goal of growing their business and improving their technology.
Another problem was moving from a prototype to a platform that was ready for production. The founders of Deepgram learned the hard way that early customers needed more than a good demo; they needed a solution that would work and grow. At first, the team tried a product-led growth (PLG) model, but they did better when they focused on selling directly to businesses. This change let them customize their products for certain use cases, which helped them gain the trust of early adopters like NASA, Spotify, and Twilio.
The Breakthrough: Changing the Way Speech Recognition Works
Deepgram’s main innovation is its end-to-end deep learning approach, which replaced old speech-to-text (STT) models with a platform that put speed, accuracy, and cost-effectiveness first. Deepgram’s models are different from other systems because they are trained on huge, varied datasets. This lets them handle different accents, languages, and noisy environments with more than 90% accuracy. Their technology can transcribe in real time with less than 300 milliseconds of delay, which makes it perfect for call centers, healthcare, and media.
The speech recognition API is the company’s most important product. It lets developers add advanced voice features to their apps. Deepgram stands out from the rest because it has features like diarization (separating speakers), summarization, sentiment analysis, and topic detection. Deepgram can run on GPUs instead of CPUs, which cuts computing costs by up to 8 times and speeds up batch transcriptions by 1200 times. This makes it a game-changer for both startups and big companies.
Deepgram shows that it cares about accessibility by supporting more than 30 languages and being able to train custom models for specific situations, like industry-specific language or unique accents. Because Deepgram is so flexible, it is a popular choice for fields like education, healthcare, and customer service.
How to Grow Your Business: The Startup Program and Beyond
Deepgram started its Startup Program in 2021 with help from accelerators like Y Combinator. The program gives early-stage companies up to $100,000 in free credits to help them make voice-enabled apps. By 2024, the program had grown to include 110 startups from a wide range of fields, such as healthcare, AR/VR, and podcast tools. The program gives you credits and special benefits, like getting new releases (like the Aura TTS model) early and direct help from Deepgram’s engineers on Slack. In the first quarter of 2024, 28 new startups joined, using up more than 2.3 million minutes of free credits.
Deepgram is growing in more ways than just its technology. The company encourages a lively community with things like the AI Minds podcast, the Weekly AI Minds newsletter, and guest speakers at office hours. These efforts give startups information on things like real-time processing and API optimization, which helps them grow in the right way.
Competing with Giants: How to Get Ahead of Hyperscalers
Deepgram has had to deal with competition from hyperscalers like Google, AWS, and OpenAI along the way. But the company has found a niche by focusing on what CEO Scott Stephenson calls “first principles” innovation. Deepgram’s platform was built from the ground up with deep learning at its core, unlike legacy systems that have been upgraded with AI. This method lets them offer better accuracy, lower latency, and prices that are good for business—achieving 85% gross margins while still being competitive.
Stephenson says that Deepgram’s strategy is like that of Tesla or SpaceX, which focuses on being flexible and coming up with new ideas rather than being big. Deepgram has outsmarted bigger companies by working with hyperscalers like NVIDIA while still being independent. They now have over 500 loyal customers.
The Future: A World Where Voice Comes First
Deepgram is more than just a speech-to-text service today; it’s a cutting-edge AI company that is changing how people and machines work together. Deepgram is ready to change the future of voice technology because it has transcribed more than 1 trillion words and has a lot of advanced features, such as text-to-speech and voice agent APIs. Deepgram Saga, their most recent product, is a voice OS for developers that shows how committed they are to making AI conversations sound like real people.
Deepgram’s story shows how powerful curiosity and hard work can be as the company keeps coming up with new ideas. Deepgram has shown that a bold vision and cutting-edge technology can make all the difference, even in a crowded market. It started as a physicist’s side project and is now a leader in voice AI. Deepgram is going to keep pushing the limits of what voice can do because its culture is based on curiosity and putting customers first.
