Senior Member of Technical Staff, Agents
加入一个小而快速发展的团队，建立自主人工智能能力以提高执行生产力。你将开发代理的能力和评估系统，帮助确保重要任务可靠完成，并且在到达用户之前证明了改进。你会做什么？- 构建完成委托工作的代理最终结束，需要最小的行政监督。- 定义并测量代理质量，以便团队可以自信地评估任务如何经常正确完成。- 为高管已经使用的工具开发新的代理功能，并在启动前评估准备性。- 调查生产故障，确定潜在原因和防止重复问题的解决方案。- 在规模上建立检测系统以发现已知故障模式和意想不到的问题。- 当工具或网站失败时改善代理恢复。- 创建可重复评估工作流程，使工程师能够衡量变化的影响。我们正在寻找什么 - 至少3年在数据科学或机器学习中发挥作用，构建或提供产品或系统的评价系统，指标或质量测量。- 生产质量的 Python SQL 和分析技能，大型数据集和自动化管道。- 经验设计评估框架，包括可测量的成功标准，代表性数据集，适当的指标和可靠基准。- 有相关影响记录，在一个角色中具有证明进展的持续经验，以及强大的工程计划学位。- 与应用人工智能或机器学习系统的经验和解决模糊问题的协作，反复方法。- 有关工具和技术包括 Python,SQL,Django,React,TypeScript,利用工程，Braintrust 和 Raindrop.补偿与福利签证基础工资：每年200,000美元至300,000美元。赞助商不提供。现在，我们在加利福尼亚州帕洛阿尔托工作。
查看英文原文
ABOUT THE ROLE
Join a small, fast-moving team building autonomous AI capabilities for executive productivity. You will develop agent capabilities and evaluation systems that help ensure important tasks are completed reliably, and that improvements are proven before reaching users.
WHAT YOU'LL DO
- Build agents that complete delegated work end to end, with minimal need for executive oversight.
- Define and measure agent quality so the team can confidently assess how often tasks are completed correctly.
- Develop new agent capabilities for the tools executives already use, and evaluate readiness before launch.
- Investigate production failures, identify underlying causes, and ship fixes that prevent repeat issues.
- Build detection systems to surface known failure patterns and unexpected issues at scale.
- Improve agent recovery when tools or websites fail.
- Create repeatable evaluation workflows that let engineers measure the impact of changes.
WHAT WE'RE LOOKING FOR
- At least 3 years in data science or machine learning roles building or delivering evaluation systems, metrics, or quality measurement for products or systems.
- Production-quality Python and SQL skills for analysis, large datasets, and automated pipelines.
- Experience designing evaluation frameworks, including measurable success criteria, representative datasets, appropriate metrics, and reliable benchmarks.
- A track record of relevant impact, sustained experience in a role with demonstrated progression, and a degree from a strong engineering program.
- Experience with applied AI or machine learning systems and a collaborative, iterative approach to solving ambiguous problems.
- Relevant tools and technologies include Python, SQL, Django, React, TypeScript, harness engineering, Braintrust, and Raindrop.
COMPENSATION & BENEFITS
Base salary: $200,000 to $300,000 annually. Visa sponsorship is not available.
LOCATION
On-site in Palo Alto, California. The team works in the office four days per week.