高级数据工程师 – 数据质量与可观测性 | 远程 1789
Senior Data Engineer – Data Quality & Observability | Remote 1789
这是一个远程职位。
你是否热衷于构建可靠、可扩展的数据平台，并在企业层面提升数据质量？我们正在寻找一名高级数据工程师——数据质量和可观测性，来领导现代数据质量框架的实施，使工程团队能够在数据问题影响业务运营之前检测、监控和预防这些问题。
在这个职位中，你将推动工程自有的数据质量实践的实施，操作GX Core，建立验证标准，并构建可观测性解决方案，以提高报告、同步流程和操作工作流中的信心。
职责
· 在整个平台上设计和实现可扩展的数据质量框架。
- 领导GX Core（Great Expectations）作为主要数据验证框架的实施和操作化。
- 使用Rule-as-Code方法开发和维护可重用的数据质量规则。
- 为关键业务数据集和工作流创建自动化验证检查。
- 实施数据可观测性、监控、警报和报告解决方案。
- 定义关键业务领域的数据血缘。
- 设计数据完整性、准确性、完整性、一致性、对账、新鲜度和异常检测的验证流程。
- 将数据质量验证集成到CI/CD流水线和发布流程中。
- 开发仪表板和报告以监控数据质量趋势和操作健康状况。
- 调查重复数据问题的根本原因并实施预防性解决方案。
- 与数据工程、应用工程、QA、产品和客服团队合作，建立数据质量的所有权和治理。
- 定义治理、验证频率、修复流程和质量指标的标准。
- 持续改进数据质量流程并建立长期的可观测性最佳实践。
要求
· 5年以上数据工程师或类似数据工程岗位的工作经验。
- 在设计和实施企业级数据质量框架方面有丰富经验。
- 有GX Core（Great Expectations）或类似工具（如Soda）的实际操作经验。
- 强大的SQL技能，以及使用Aurora PostgreSQL和Amazon Redshift的经验。
- 有设计数据验证规则、对账流程和可观测性解决方案的经验。
- 有构建和维护ETL流水线和大规模数据工作流的经验。
·
查看英文原文
This is a remote position.
Are you passionate about building reliable, scalable data platforms and improving data quality at an enterprise level? We are looking for a Senior Data Engineer – Data Quality & Observability to lead the implementation of a modern data quality framework, enabling engineering teams to detect, monitor, and prevent data issues before they impact business operations.
In this role, you'll drive the implementation of engineering-owned data quality practices, operationalize GX Core, establish validation standards, and build observability solutions that improve confidence across reporting, synchronization processes, and operational workflows.
Responsibilities
· Design and implement a scalable data quality framework across the platform.
- Lead the implementation and operationalization of GX Core (Great Expectations) as the primary data validation framework.
- Develop and maintain reusable data quality rules using a Rule-as-Code approach.
- Create automated validation checks for business-critical datasets and workflows.
- Implement data observability, monitoring, alerting, and reporting solutions.
- Define and maintain data lineage across key business domains.
- Design validation processes for data completeness, accuracy, integrity, consistency, reconciliation, freshness, and anomaly detection.
- Integrate data quality validations into CI/CD pipelines and release processes.
- Develop dashboards and reports to monitor data quality trends and operational health.
- Investigate root causes of recurring data issues and implement preventive solutions.
- Collaborate with Data Engineering, Application Engineering, QA, Product, and Support teams to establish ownership and governance for data quality.
- Define standards for governance, validation frequency, remediation workflows, and quality metrics.
- Continuously improve data quality processes and establish long-term observability best practices.
Requisitos
- 5+ years of experience as a Data Engineer or in similar data engineering roles.
- Strong experience designing and implementing enterprise Data Quality frameworks.
- Hands-on experience with GX Core (Great Expectations) or similar tools such as Soda.
- Strong SQL skills and experience working with Aurora PostgreSQL and Amazon Redshift.
- Experience designing data validation rules, reconciliation processes, and observability solutions.
- Experience building and maintaining ETL pipelines and large-scale data workflows.
- Strong understanding of data modeling, referential integrity, synchronization, and batch processing.
- Experience integrating data validation into CI/CD pipelines.
- Experience with Git and engineering best practices such as Rule-as-Code.
- Experience building dashboards, alerts, and reporting for operational monitoring.
- Strong analytical and problem-solving skills with experience performing root cause analysis.
- Experience collaborating with cross-functional engineering teams.
- Excellent communication and documentation skills.
What We Offer
· ? Fully remote position.
- ? Opportunity to build enterprise-scale data quality and observability solutions.
- ? High-impact role with ownership over data quality strategy and engineering best practices.
- ? Collaborative environment working alongside Data Engineering, QA, Product, and Application Engineering teams.
- ? Opportunity to work with modern data validation, observability, and cloud data technologies while driving continuous improvement across the platform.
Originally posted on Himalayas