高级软件工程师 - 模型评估与AI系统
Senior Software Engineer - Model Evaluation & AI Systems
## **公司简介**
Deepgram 是支撑新兴万亿美元语音 AI 经济的领先平台，提供实时 API 用于语音转文本（STT）、文本转语音（TTS），并可大规模构建生产级语音代理。超过 20 万名开发者和 1,300 多家组织正在构建“由 Deepgram 提供支持”的语音产品，包括 Twilio、Cloudflare、Sierra、Decagon、Vapi、Daily、Cresta、Granola 和 Jack in the Box。Deepgram 的语音原生基础模型可通过云 API 或自托管及本地软件访问，具有无与伦比的准确性、低延迟和成本效率。由领先的全球投资者和战略合作伙伴牵头的最新 C 轮融资支持，Deepgram 已处理超过 5 万年的音频，并转录了超过 1 万亿个单词。世界上没有哪家公司比 Deepgram 更了解语音。
## **公司运营节奏**
在 Deepgram，我们期望具备 AI 首先的思维模式——AI 的使用和熟悉度不是可选的，而是我们运作、创新和衡量绩效的核心。
每一位在 Deepgram 工作的团队成员都应积极使用和试验先进的 AI 工具，甚至将它们构建到日常工作中。我们衡量 AI 在交付结果方面的应用效果，持续且富有创意地使用最新 AI 能力是这里成功的关键。候选人应能够快速采用新模型和新模式，将 AI 整合到自己的工作流程中，并不断推动这些技术的边界。
此外，我们以 AI 的速度前进。变化迅速，你每天的工作内容也会随之快速变化。如果你对实验、适应、临场思考和不断学习不感兴趣，或者你寻求的是高度规范化的传统朝九晚五工作，那么这可能不是适合你的职位。
### **职位机会**
Deepgram 正在寻找一名高级软件工程师 - 模型评估与 AI 系统，加入负责在我们的语音、音频和多语言模型交付给客户前验证其质量的团队。
该团队负责评估和质量保障的界面，确保 Deepgram 的模型——涵盖语音转文本、文本转语音，以及越来越多的 LLM 和多模态驱动的系统——在批量和流式环境中都能达到性能目标。我们构建管道、测试框架、监控工具和 Canary 工具，以捕捉回归问题、幻觉和质量问题。
查看英文原文
## **Company Overview**
Deepgram is the leading platform underpinning the emerging trillion-dollar Voice AI economy, providing real-time APIs for speech-to-text (STT), text-to-speech (TTS), and building production-grade voice agents at scale. More than 200,000 developers and 1,300+ organizations build voice offerings that are ‘Powered by Deepgram’, including Twilio, Cloudflare, Sierra, Decagon, Vapi, Daily, Cresta, Granola, and Jack in the Box. Deepgram’s voice-native foundation models are accessed through cloud APIs or as self-hosted and on-premises software, with unmatched accuracy, low latency, and cost efficiency. Backed by a recent Series C led by leading global investors and strategic partners, Deepgram has processed over 50,000 years of audio and transcribed more than 1 trillion words. There is no organization in the world that understands voice better than Deepgram.
## **Company Operating Rhythm**
At Deepgram, we expect an AI-first mindset—AI use and comfort aren’t optional, they’re core to how we operate, innovate, and measure performance.
Every team member who works at Deepgram is expected to actively use and experiment with advanced AI tools, and even build your own into your everyday work. We measure how effectively AI is applied to deliver results, and consistent, creative use of the latest AI capabilities is key to success here. Candidates should be comfortable adopting new models and modes quickly, integrating AI into their workflows, and continuously pushing the boundaries of what these technologies can do.
Additionally, we move at the pace of AI. Change is rapid, and you can expect your day-to-day work to evolve just as quickly. This may not be the right role if you’re not excited to experiment, adapt, think on your feet, and learn constantly, or if you’re seeking something highly prescriptive with a traditional 9-to-5.
### **The Opportunity**
Deepgram is looking for a Senior Software Engineer - Model Evaluation & AI Systems to join the team responsible for validating the quality of our speech, audio, and multilingual models before they reach customers.
This team owns the evaluation and quality assurance surfaces that ensure Deepgram's models — across speech-to-text, text-to-speech, and increasingly LLM- and multimodal-powered systems — meet their performance targets in both batch and streaming environments. We build the pipelines, harnesses, canaries, and test frameworks that catch regressions, hallucinations, and quality issues before they impact customers, and we partner closely with Research to turn model expectations into automated, reproducible, enforceable checks.
In this role, you'll define evaluation methodology and build the infrastructure that measures model quality at scale. You'll create evaluation pipelines, define pass/fail criteria grounded in Research benchmarks, and build the monitoring that keeps models honest in production. Your work provides the trusted signals that inform release and optimization decisions, and directly protects the customer experience.
We're looking for a strong engineer who is equally comfortable building test infrastructure and reasoning about model behavior. This role is aimed at senior engineers with broad instincts for quality, measurement, and automation; hands-on experience evaluating modern AI systems is a strong plus.
### **What You'll Do**
- Define and build evaluation methodologies for Deepgram's models, spanning speech-to-text, text-to-speech, and emerging LLM, RAG, agent, and multimodal systems.
- Design, build, and maintain automated evaluation pipelines across batch and streaming (e.g. WER, runaway/hallucination detection, latency and time-to-first-byte), with a focus on correctness, reproducibility, and ease of adoption.
- Build scalable, reproducible evaluation infrastructure — harnesses, orchestration, and result-aggregation pipelines — running against production models and, where needed, large GPU clusters.
- Translate Research benchmarks and expected model metrics into automated, enforceable pass/fail gates.
- Build and operate canaries and continuous-monitoring systems that detect quality regressions in production before they reach customers.
- Partner with DevOps/Infra to stand up ephemeral test environments and results-aggregation infrastructure.
- Work alongside Research, model training, inference, and product teams to provide trusted evaluation signals that inform release and optimization decisions.
- Integrate evaluation and quality gates into CI/CD so quality is verified continuously, not manually.
- Help raise the bar through code reviews, technical design discussions, and strong engineering and QA practices.
### **What We're Looking For**
- BS, MS, or PhD in Computer Science, AI, Applied Math, or a related field, or equivalent experience.
- 5+ years of professional software or QA engineering experience, with a track record of shipping test infrastructure or evaluation systems (senior candidates with significantly deeper experience welcome).
- Solid backend/scripting experience in a language such as Python, Rust, Go, or similar.
- Experience designing and building automated test pipelines, evaluation frameworks, or data-processing systems.
- Strong analytical skills and comfort reasoning about metrics, thresholds, and statistical variation in results — able to distinguish real regressions from noise.
- Ability to take charge of ambiguous technical challenges and communicate effectively across research, engineering, and product teams.
### **Nice to Have / Ways to Stand Out**
- Hands-on experience evaluating modern AI systems such as LLMs, RAG pipelines, agents, or multimodal models, including model behavior analysis.
- Experience with React Native or other cross-platform mobile frameworks for building tooling that's accessible beyond the desktop.
- Experience building or improving evaluation frameworks, benchmarks, or ML infrastructure used by other teams or external users.
- A strong appreciation for evaluation quality — correctness, reproducibility, and consistency across environments.
- Experience with voice, audio, speech recognition, or real-time systems, and familiarity with metrics like WER, MOS, or latency/TTFB.
- Prior involvement in open-source projects, through contributions, reviews, maintenance, or community engagement.
- Experience acting as a technical bridge across teams or platforms (evaluation, training, inference, agent frameworks), combining architectural understanding with clear communication and influence.
- Familiarity with cloud infrastructure, containerized/ephemeral environments, and monitoring tooling (e.g. Grafana, canaries, anomaly detection).
_**Notice**: We're aware of individuals impersonating Deepgram recruiters. All legitimate Deepgram recruiting communication comes from an @_ [_deepgram.com_](http://deepgram.com) _email address. If you've received a message claiming to be Deepgram, please forward it to careers@deepgram.com._