Senior Data Engineer – Data Quality & Observability | Remote 1789
这是一个远程职位。你是否热衷于建立可靠,可扩展的数据平台和企业级别提高数据质量?我们正在寻找一名高级数据工程师 数据质量与观察能力领导实施现代数据质量框架,使工程团队能够在影响商业运营之前检测,监控和预防数据问题。在这个角色中,你将推动工程拥有数据质量实践的实现,操作 GX 核心,制定验证标准,并构建可观测性解决方案,以改善报告,同步流程和运行工作流程中的信心。· 设计和实施可扩展的数据质量框架。· 领导 GX 核心 (大期望)的实现和运行作为主要数据验证框架。· 使用规则为代码方法制定和维护可重复使用的数据质量的规则。· 为关键业务数据集和工作流程创建自动验证检查。· 实施数据可观察,监测,警报和报告解决方案。· 在关键业界中定义并维持数据系。· 设计对数据完整性,准确性,完整性،一致性,和解,新鲜度和异常检测的验证过程。· 集成数据质量验证到 CI/CD 管道和发布流程中。·开发仪表板和报告来监测数据质量趋势和运营健康。·调查反复出现的数据问题根源原因并实施预防性解决方案。·与数据工程,应用工程,QA,产品和支持团队合作建立对数据质量的所有权和治理。·定义治理标准,验证频率,修复工作流程和质量指标。·不断改善数据质量流程并建立长期可观察性的最佳实践。要求 · 5年以上作为数据工程师或类似数据工程角色的经验。· 强大的经验设计和实施企业数据质量框架。· GX Core (Great Expectations) 或 Soda 等类似工具的实践经验。· 强大的 SQL 技能和与 Aurora PostgreSQL 和 Amazon Redshift 合作经验。• 设计数据验证规则,调整过程和可观察性解决方案的经验。
查看英文原文
This is a remote position.
Are you passionate about building reliable, scalable data platforms and improving data quality at an enterprise level? We are looking for a Senior Data Engineer – Data Quality & Observability to lead the implementation of a modern data quality framework, enabling engineering teams to detect, monitor, and prevent data issues before they impact business operations.
In this role, you'll drive the implementation of engineering-owned data quality practices, operationalize GX Core, establish validation standards, and build observability solutions that improve confidence across reporting, synchronization processes, and operational workflows.
Responsibilities
· Design and implement a scalable data quality framework across the platform.
- Lead the implementation and operationalization of GX Core (Great Expectations) as the primary data validation framework.
- Develop and maintain reusable data quality rules using a Rule-as-Code approach.
- Create automated validation checks for business-critical datasets and workflows.
- Implement data observability, monitoring, alerting, and reporting solutions.
- Define and maintain data lineage across key business domains.
- Design validation processes for data completeness, accuracy, integrity, consistency, reconciliation, freshness, and anomaly detection.
- Integrate data quality validations into CI/CD pipelines and release processes.
- Develop dashboards and reports to monitor data quality trends and operational health.
- Investigate root causes of recurring data issues and implement preventive solutions.
- Collaborate with Data Engineering, Application Engineering, QA, Product, and Support teams to establish ownership and governance for data quality.
- Define standards for governance, validation frequency, remediation workflows, and quality metrics.
- Continuously improve data quality processes and establish long-term observability best practices.
Requisitos
- 5+ years of experience as a Data Engineer or in similar data engineering roles.
- Strong experience designing and implementing enterprise Data Quality frameworks.
- Hands-on experience with GX Core (Great Expectations) or similar tools such as Soda.
- Strong SQL skills and experience working with Aurora PostgreSQL and Amazon Redshift.
- Experience designing data validation rules, reconciliation processes, and observability solutions.
- Experience building and maintaining ETL pipelines and large-scale data workflows.
- Strong understanding of data modeling, referential integrity, synchronization, and batch processing.
- Experience integrating data validation into CI/CD pipelines.
- Experience with Git and engineering best practices such as Rule-as-Code.
- Experience building dashboards, alerts, and reporting for operational monitoring.
- Strong analytical and problem-solving skills with experience performing root cause analysis.
- Experience collaborating with cross-functional engineering teams.
- Excellent communication and documentation skills.
What We Offer
· ? Fully remote position.
- ? Opportunity to build enterprise-scale data quality and observability solutions.
- ? High-impact role with ownership over data quality strategy and engineering best practices.
- ? Collaborative environment working alongside Data Engineering, QA, Product, and Application Engineering teams.
- ? Opportunity to work with modern data validation, observability, and cloud data technologies while driving continuous improvement across the platform.
Originally posted on Himalayas