Senior Deep Learning Engineer, Accuracy Evaluation
我们正在寻找高级工程师，以先进地评估创新深度学习模型的性能和能力。包括 LLM,RAG,代理人和视觉模型。你将在整个组织中合作，让我们社区和伙伴的最新旗舰模型生效。这个角色提供了一个杰出机会来创造人工智能未来。AI 革命前沿的一家快速增长的公司。加入我们的世界级软件工程师和合作伙伴团队，以提供最先进的模型与闪电速度推断。你将在拥有数百个 PetaFLOPS 的企业级 GPU 集群上工作，并获得早期访问未发布硬件，直接影响 NVIDIA 路线图和更广泛的人工智能景观！你将做什么：·为 NVIDIA 的边界模型设计和构建决策等级评估环境，涵盖推理，多模特，长文本和代理系统，产生可审核准确信号，进入每个主要模型版本。· 研发和开发新型模型家庭和能力领域 （低精度数值，多轮代理任务，代码生成）的新的评估方法。建立并运营评估基础设施和管道，包括基准环境，回归 CI 系统和统计分析工具，用于 NVIDIA 各地的模型，产品和应用研究团队。· 与模型研究，培训和客户团队合作，将评估信号转化为具体决策：释放/不使用，培训反复方向以及对外部边界模型进行竞争性定位。我们需要看到的：· 计算机科学，机器学习，统计学或相关领域的 BS,MS 或博士学位 · 6年以上与 LLM 实践经验，设计和运行大型语言模型或多模式人工智能系统的评估，包括使用代理，多转移或推理重设置的经验。·强大的统计基础：实验设计，意义测试，回归分析以及在规模上区分标志结果中的噪音的能力。·已证明的经验不仅是利用现有标志 （管道，标志带，可复制的 CI 系统）的评价基础设施建设。· 清晰，精确的沟通者能够将量化评估结果转换为决策
查看英文原文
We are seeking senior engineers to pioneer new methodologies for accurately assessing the performance and capabilities of ground-breaking deep learning models, including LLMs, RAG, agents, and vision models. You will collaborate across the organization to bring the latest flagship models from our community and partners to life. This role offers an outstanding opportunity to craft the future of AI at a fast-growing company at the forefront of the AI revolution. Join our team of world-class software engineers and partners to deliver the most advanced models with lightning-fast inference. You'll work on the most powerful, enterprise-grade GPU clusters capable of hundreds of PetaFLOPS and gain early access to unreleased hardware, making a direct impact on NVIDIA's roadmap and the broader AI landscape!
What you’ll be doing:
- Design and build decision-grade evaluation environments for NVIDIA's frontier models spanning reasoning, multimodal, long-context, and agentic systems, producing auditable accuracy signals that gate every major model release.
- Research and develop novel evaluation methodologies for emerging model families and capability domains (low-precision numerics, multi-turn agentic tasks, code generation) where established benchmarks don't yet exist or don't generalize.
- Build and operate the evaluation infrastructure and pipelines including benchmark environments, regression CI systems, and statistical analysis tooling, used by model, product, and applied research teams across NVIDIA.
- Partner with model research, training, and customer teams to translate evaluation signals into concrete decisions: release go/no-go, training iteration direction, and competitive positioning against external frontier models.
What we need to see:
- BS, MS, or PhD in Computer Science, Machine Learning, Statistics, or a related field
- 6+ years of hands-on experience with LLMs, designing and running evaluations for large language models or multimodal AI systems, including experience with agentic, multi-turn, or reasoning-heavy settings.
- Strong statistical foundations: experimental design, significance testing, regression analysis, and the ability to distinguish signal from noise in benchmark results at scale.
- Proven experience building evaluation infrastructure (pipelines, benchmark harnesses, reproducible CI systems) not just consuming existing benchmarks.
- Clear, precise communicator who can translate quantitative evaluation results into decisions for researchers, product teams, and senior leadership.
Ways to stand out from the crowd:
- Deep familiarity with open-source evaluation frameworks
- Experience designing evaluations for agentic systems: tool use, multi-turn reasoning, environment-based benchmarks (SWE-bench, GAIA, WebArena-style), or interactive evaluation settings.
- Track record of publishing or contributing to evaluation research, new benchmark design, methodology papers, or reproducibility analyses that shaped how the field measures model capability.
- Experience measuring model accuracy under low-precision inference (FP8, INT4, quantization-aware settings) and understanding how calibration and sparsity interact with benchmark results.
- Comfort running large-scale workloads on HPC/Slurm clusters, including reproducible experiment management (MLflow, W&B) and compute cost optimization across hundreds of benchmark runs.
NVIDIA is committed to fostering a diverse work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.
Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. For Poland: The base salary range is 375,000 PLN - 650,000 PLN.Originally posted on Himalayas