QA工程师
QA Engineer
DEFCON AI 是一家洞察公司，利用人工智能、数学优化、数据分析和软件工程，对复杂系统进行弹性优化。
在当今不断变化的世界中，DEFCON AI 的技术能够使结果与运营目标保持一致，提升决策质量，并赋能客户预测、评估和减轻中断的影响。
**职位描述**
作为 QA 工程师，你将负责一个 AI 驱动的决策支持系统的质量：测试计划、自动化覆盖范围，以及证明所交付的内容符合规格的证据。产品定义了验收标准。**你将构建其可执行版本**，并确保工程、平台和其他交付工作的团队对此负责。
**该职位位于组织的产品侧，而非工程侧。** 你将通过产品负责人汇报，并根据产品方向开展工作，同时每天与工程和平台团队协作。嵌入在工程团队中的测试人员通常验证已构建的内容，因为他们看到的是实现。而该职位验证的是已指定的内容。从代码编写的测试套件可能通过，但可能遗漏了客户的需求。
这种汇报关系并不降低工程标准。你将编写代码并构建真实的自动化测试，而不是管理测试用例，你需要对应用代码有足够熟练的掌握以进行有效测试。作为首位专职的质量工程人员，你将建立方法而非继承现有流程，因此我们希望你对测试应该如何进行有清晰的观点。我们使用当前的工具进行开发，并期望你同样使用这些工具，包括在我们自己的工程实践中使用 AI 协助。
这是一个完全远程的职位，偶尔需要出差（最多 25%），包括前往 DEFCON AI 总部、客户现场和供应商设施。
**主要职责**
**测试策略和验收标准**
- 负责系统测试计划，涵盖功能、集成、性能和负载测试，并将其作为正式交付物进行维护
- 将产品设定的验收标准转化为可执行的测试
- 在开发开始前与产品团队一起审查验收标准，并识别其中模糊或无法验证的要求
- 对工程、平台和合作伙伴团队的验收标准负责，并定义在变更被考虑之前必须存在的证据
查看英文原文
ABOUT DEFCON AI
RESILIENCE IN THE FACE OF DISRUPTION. DEFCON AI is an insights company that leverages artificial intelligence, mathematical optimization, data analytics, and software engineering for resilient optimization of complex systems.
In today’s dynamically changing world, DEFCON AI’s technology aligns outcomes with operational goals, better decision making, and empowers customers to anticipate assess, and mitigate the impacts of disruptions.
**About the Role**
As QA Engineer you will own quality for an AI-enabled decision-support system: the test plan, the automated coverage, and the evidence that what was delivered is what was specified. Product defines the acceptance criteria. **You build the executable version of it**, and you hold engineering, platform, and everyone else delivering the work accountable to it.
**This role sits on the product side of the organization, not the engineering side.** You will report through the Product Lead and take direction from product, while working with engineering and platform every day. A tester embedded in engineering tends to verify what was built, because the implementation is what they see. This role verifies what was specified. A test suite written from the code will pass while still missing what the customer asked for.
That reporting line does not lower the engineering bar. You will write code and build real automation rather than managing test cases, and you will need enough fluency in the application code to test it well. As the first dedicated quality engineering hire you are establishing the approach rather than inheriting one, so we want someone with a clear point of view on how testing should work. We build with current tooling and expect the same, including AI assistance in our own engineering practice.
This is a fully remote role with occasional travel (up to 25%) to DEFCON AI HQ, customer sites, and vendor facilities as required.
**Key Responsibilities**
**Test Strategy and Acceptance Criteria**
- Own the system test plan covering functional, integration, performance, and load testing, and maintain it as a formal deliverable
- Translate the acceptance criteria set by product into executable tests
- Review acceptance criteria with product before development begins, and identify requirements that are ambiguous or not verifiable as written
- Hold engineering, platform, and partner teams accountable to the acceptance criteria, and define what evidence must exist before a change is considered complete
- Work alongside the engineering team day to day, close enough to test their work well and independent enough to report what the testing shows
- Plan and coordinate user acceptance testing with the customer, and track findings through to resolution
**Test Automation and Coverage**
- Build and maintain automated test suites, and integrate them into the CI/CD pipeline as release gates
- Establish and hold an automated coverage standard, and report against it
- Build performance and load testing that validates the system under its expected user load
- Maintain regression coverage across releases
**Measurement and Evidence**
- Build and maintain labeled evaluation sets, working with subject-matter experts whose time is limited
- Design and run sampled audits confirming that automated decisions were correct
- Measure and report what the system misses alongside what it gets right
- Measure how often users override the system, and report the trend
- Document how each committed measure is computed, on what data, and what evidence accompanies a release
- Produce this evidence independently of the people building the models
**Release Governance**
- Maintain the version inventory across builds, models, configurations, and thresholds
- Produce test reports, evaluation records, and rollback criteria for each release
- Work with platform engineering to confirm that test environments match production closely enough for results to be meaningful
- Hold the gate: no release ships without the evidence its acceptance requires
**Required Qualifications**
- **5+ years** in software quality engineering, test automation, or software engineering with substantial dedicated test ownership
- **A clear point of view on how testing should be done**, and the judgment to know which practices are worth holding to. You will be setting the standard here rather than following one
- **Strong coding ability in a language and framework used for test automation**, for example Python with pytest, or JavaScript or TypeScript with Playwright, Cypress, or Selenium. Python is what most of this stack is written in. You should also be able to read and reason about the application code under test
- **Routine use of AI-assisted development**, with informed judgment about where it adds value and where its output requires verification. We expect AI tooling to be part of how test coverage gets built here, not an occasional experiment
- Demonstrated experience building automated test suites and integrating them into CI/CD as release gates
- Experience owning a test plan through review and acceptance
- **Demonstrated habit of testing against the requirement rather than the implementation**, including working directly with product owners or end users to establish what correct actually means
- Experience with performance and load testing, including establishing what a result does and does not demonstrate
- Working statistical literacy, including sampling design and the confidence a measurement can support
- Experience measuring what a system misses, not only what it gets right
- **Willingness to report a result the team did not want**, and to keep the measurement independent of the people whose work it evaluates
- **Ability to gate work owned by people who do not report to you**, and to earn enough credibility with engineers that the gate holds
- Clear, audit-ready written documentation, since much of the output of this role is evidence that others read
- Willingness to hold a release, and explain the reason, to people who want it shipped
- US Citizenship Required
- **Active US Secret clearance.** The work is performed in a controlled government cloud environment and requires a favorable investigation and CAC eligibility from the start
- Elevated security requirements apply to portions of this work and are discussed during the interview process
- Willingness to travel up to 25% to customer sites, DEFCON AI HQ, and vendor facilities as required
**Preferred Qualifications**
- **Clearance:** active Top Secret
- **Environment:** government DevSecOps, RMF, or ATO environments, and experience producing test evidence for a formal accreditation or audit
- **Automation:** contract and API testing, test data management, and building test infrastructure others can extend
- **AI-assisted testing:** using language models to generate and maintain test coverage, and the practices that keep generated tests meaningful rather than merely passing
- **Performance:** JMeter, k6, Locust, or comparable
- **Evaluation:** validating machine learning or model output, including groundedness and citation checking, and analyzing where users disagreed with a system recommendation
- **Domain:** systems where an incorrect result carries real cost, and where testing had to satisfy an outside reviewer rather than only an internal one
- **Governance:** model cards, NIST AI RMF, or comparable responsible AI practice
- **Accessibility:** Section 508 and WCAG verification
**What Success Looks Like**
- A test plan the customer accepts, and that holds up as the system grows
- Acceptance criteria that are verifiable before development starts
- Automated coverage meeting its standard and running as a release gate
- Sampled audits designed to catch an incorrect automated decision
- Performance evidence that reflects realistic load
- A release trail complete enough to reconstruct why any version shipped
**What We Offer:**
- A fully remote, results-based environment
- Competitive salary, bonus, and equity package
- 100% employer paid, comprehensive health insurance including medical, dental, and vision for you and your family
- Unlimited PTO, with your manager’s approval
- Flexible work environment where you manage your work day
- 14 weeks of fully-paid parental leave
**Salary Range:** $135,000-$170,000. This represents the typical salary range for this position based on experience, skills, and other factors.
We’re an Equal Opportunity Employer: You’ll receive consideration for employment without regard to race, sex, color, religion, sexual orientation, gender identity, national origin, protected veteran status, or on the basis of disability.
* * *
**_Applicant Data Disclosure_**
_By submitting an application, you acknowledge that Defcon AI uses third-party service providers to facilitate its recruitment and hiring processes. These providers include applicant tracking systems, candidate verification platforms, and fraud detection tools (collectively, "Hiring Platforms"). Your application materials, including your résumé, cover letter, work samples, responses to application questions, and any other information you submit, may be transmitted to and processed by these Hiring Platforms for the following purposes:_
- _Managing and administering your application throughout the hiring process;_
- _Verifying the accuracy and authenticity of application materials, including by cross-referencing information you provide against publicly available sources and proprietary databases;_
- _Identifying indicators of potentially fraudulent, fabricated, or materially misleading application content, including but not limited to discrepancies between submitted materials and publicly available professional profiles, geographic anomalies, and fabricated work histories._
_Applications that are flagged through this process as containing indicators of fraud or material misrepresentation may be declined from further consideration. If you have questions about the status of your application or the evaluation process, please contact [recruiting@defconai.com](mailto:recruiting@defconai.com)._
_Defcon AI requires its Hiring Platform providers to process your information solely for the purposes described above and in accordance with applicable law. Your information will be retained only for as long as necessary to fulfill these purposes and any applicable legal obligations, after which it will be deleted in accordance with Defcon AI's data retention policies._
_For more information about how your data is used, please refer to our Privacy Policy and_ _[Applicant Privacy Notice](https://nam09.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redcellpartners.com%2Fapplicant-privacy-policy%2F&data=05%7C02%7Ckat.creamer%40redcellpartners.com%7Cc0f94f3daed94dc7503108de8b61955e%7Cf861de501a2a42359dbf28afd57d1d97%7C0%7C0%7C639101448841578570%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=WoPiBMZqQyPPK5N4E1sCagkbMY2S8P3aSRSoy7T4los%3D&reserved=0)_ _._