软件工程总监，代理评估
Software Engineering Director, Agentic Evaluations
**关于G2 - 公司**
G2是全球最大的可信软件市场。当你加入G2时，你将加入行业领先的团队，通过真实软件用户的可信洞察来帮助 businesses 实现其最大潜力。
现在，我们与[Capterra](https://www.capterra.com/)、[SoftwareAdvice](https://www.softwareadvice.com/)和[GetApp](https://www.getapp.com/)携手合作，打造了最大的在线数据和软件洞察来源，以推动人工智能时代的智能购买。我们拥有超过2亿的年访问量和600万条经验证的评论，现在已成为软件买家做出更好、更快决策的中心场所。
而我们才刚刚开始！我们正致力于改变全球B2B软件行业，成为人工智能时代软件买家和卖家最值得信赖的数据基础。
这听起来是否令人兴奋？加入我们，一起迈向下一个高峰！
**关于G2 - 我们的人**
在G2，我们的一切以及我们的所作所为都植根于我们的PEAK价值观——（P）绩效 + （E）企业家精神 + （A）真实性 + （K）善良。在G2工作意味着你是价值驱动、不断成长的全球社区的一部分，我们一起攀登高峰。我们为彼此的成功欢呼，从错误中学习，并在困难时期互相支持和依靠。我们怀揣雄心和企业家精神，激励彼此承担具有挑战性的工作，这将帮助我们共同成长和学习。
你将加入一个由聪明、专注且善良的个体组成的全球多元化团队，每个人都有独特的才能、抱负和人生经历。我们社区和文化的核心是我们由员工主导的ERGs，它们庆祝并突出我们全球团队的多样身份。作为一家公司，我们有意地关注我们的[DEI](https://company.g2.com/dei)和慈善工作（如我们的[G2 Gives](https://company.g2.com/gives)项目），因为这鼓励我们所有人成为更好的人。
### 关于该职位
**职责概要：**
作为软件工程总监，代理评估，你的职责将围绕提供软件供应商提供的代理的有用且可信的评估。你将增强执行代理性能验证的系统，并领导其核心团队。你将采用“以代理为中心”的工作方式，应用代理工程技巧来完成你的工作。
查看英文原文
**About G2 - The Company**
G2 is the world's largest and most trusted software marketplace. When you join G2, you’re joining the industry’s leading team that helps businesses reach their peak potential by powering decisions and strategies with trusted insights from real software users.
Now, we have joined forces with [Capterra](https://www.capterra.com/), [SoftwareAdvice](https://www.softwareadvice.com/), and [GetApp](https://www.getapp.com/) to create the largest source of online data and software insights to fuel intelligent buying in the age of AI. With 200M+ combined annual visitors and 6M verified reviews, we are now the centralized place to enable software buyers to make better and faster decisions with confidence.
And we are just getting started! We are setting out to transform the global B2B software industry and become the most trusted data foundation for buyers and sellers of software for the age of AI.
Does that sound exciting to you? Come join us as we try to reach our next PEAK!
**About G2 - Our People**
At G2, everything we are and what we do is grounded in our PEAK values— ( _P_ erformance + _E_ ntrepreneurship + _A_ uthenticity + _K_ indness. Working at G2 means you are part of a value-driven, growing global community that climbs PEAKs together. We cheer for each other’s successes, learn from our mistakes, and support and lean on one another during challenging times. With ambition and entrepreneurial spirit we push each other to take on challenging work, which will help us all to grow and learn.
You will be part of a global, diverse team of smart, dedicated, and kind individuals - each with unique talents, aspirations, and life experiences. At the heart of our community and culture are our people-led ERGs, which celebrate and highlight the diverse identities of our global team. As an organization, we are intentional about our [DEI](https://company.g2.com/dei) and philanthropic work (like our [G2 Gives](https://company.g2.com/gives) program) because it encourages us all to be better people.
### About The Role
**Summary of Responsibilities:**
As a Software Engineering Director, Agentic Evaluations, your responsibilities will center around delivery of useful and credible evaluations of agents offered by software vendors. You’ll enhance the system that performs validation of agent performance, and lead the core team behind it. You will leverage “agent-first” ways of working, applying agentic engineering techniques to deliver your work to production.
#### **Detailed Responsibilities:**
_**Build and enhance agent evaluations that run against live agents [80%]:**_
- Help scope out the feasibility and effort involved in evaluating an agent offered by a software vendor, identifying the most practical path to reaching a credible evaluation
- Own the full stack of the core evaluation system, from admin & API surface to eval workflow dispatch
- Design evaluation system primitives that generalize across software verticals, so onboarding a new category benefits from reuse
- Distill the appropriate and repeatable processes in onboarding and maintaining integrations into repeatable AI skills or agents to increase evaluation velocity
- Keep tabs on emerging frameworks and techniques for agent evaluation which offer improvements to evaluations, and provide industry-accepted means to distribute eval results
- Collaborate with data science peers in their pursuit to develop rich and proprietary benchmarks
_**Lead a group of engineers and be a source of wider influence [20%]:**_
- You will facilitate the growth and mentorship of a core team of engineers, and be a source of technical leadership and subject matter expertise in evals for them
- You will also help socialize the use of evals in agent-oriented products built by the wider organization.
### Minimum Qualifications:
We realize applying for jobs can feel daunting at times. Even if you don’t check all the boxes in the job description, we encourage you to apply anyway.
- 10+ years of professional programming experience in backend or full-stack environments
- 2+ years of experience directly managing engineers
- Expert-level proficiency in backend development using languages like Python, Java/Kotlin, Typescript/Javascript or Go; strong proficiency in associated backend frameworks like FastAPI or [Node.js](http://node.js)
- Direct experience creating evaluations (evals) against a customer-facing agent, leveraging agent trajectory trace data and rubrics to measure an agent’s task completion rate, accuracy, correctness, or policy adherence
- Direct experience using frontier models from OpenAI, Anthropic, or Google in LLM-as-a-judge applications
- Regular use of coding agent harnesses like Claude Code, Codex, Opencode, or Pi as part of the daily development workflow
- BS/BA degree in related field
#### What Can Help Your Application Stand Out:
- Experience with agent tool use via direct integration or as mediated by MCP servers
- Knowledge of STATE-Bench, tau2-bench or similar benchmark frameworks
- Experience using Playwright, browser-use, Chrome DevTools MCP or others to drive user flows in the browser
- Experience with durable execution frameworks & workflow solutions, like Temporal, DBOS, Cloudflare / Vercel Workflows, or similar; or experience with agent sandboxing using AWS E2B, Daytona, Cloudflare / Vercel Containers or similar
**Our Commitment to Inclusivity and Diversity**
At G2, we are committed to creating an inclusive and diverse environment where people of every background can thrive and feel welcome. We consider applicants without regard to race, color, creed, religion, national origin, genetic information, gender identity or expression, sexual orientation, pregnancy, age, or marital, veteran, or physical or mental disability status. Learn more about our commitments [here](https://culture.g2.com/commitments).
--
For job applicants in California, the United Kingdom, and the European Union, please review [this applicant privacy notice](https://legal.g2.com/applicant-privacy-notice) before applying to this job.
**How We Use AI Technology in Our Hiring Process**
G2 incorporates AI-powered technology to enhance our candidate evaluation process. These tools may assist with initial application screening, skills assessment analysis, and identifying candidates whose qualifications align with specific role requirements. While AI technology supports our recruitment workflow, all final hiring decisions remain under human oversight and judgment.
**Your Choice Matters:** If you would prefer that your application be reviewed without AI assistance, you can opt out by entering your email address in the email entry field at the bottom of the Automated Processing Legal Notice. Choosing to opt out will not disadvantage your application in any way—we will ensure your materials receive a thorough manual review by our hiring team.
For additional details about how we handle your information throughout the application process, please review [G2's Applicant Privacy Notice](https://legal.g2.com/applicant-privacy-notice).