Azure Data Engineer
必须具备：认证：要求 – Microsoft Certified: Azure Data Engineer Associate（或更高级别），Azure Data Factory
职位简介：
我们正在寻找一位技能娴熟的数据工程师，具备在 Azure Data Factory、Azure Databricks、Oracle 和 PostgreSQL 方面的专业知识。理想的候选人将设计和实现稳健的数据管道，优化数据流，并利用微软 Azure 生态系统参与我们的云数据平台项目。
主要职责：
- 使用 Azure Data Factory 和 Databricks（基于 Spark）设计、构建和维护可扩展的数据管道。
- 从 Oracle 和 PostgreSQL 数据库等多种数据源中获取、转换和编排数据。
- 与数据分析师、数据科学家和其他利益相关者合作，了解数据需求并提供干净、可靠的数据。
- 优化数据工作流的性能，特别是在 Azure 环境中。
- 实现数据质量检查、数据血缘追踪和监控流程。
- 遵循并贡献于数据治理、安全和合规政策。
- 参与代码审查、文档编写和持续改进计划。
所需技能与经验：
- 熟练掌握 Azure Data Factory 和 Azure Databricks（优先使用 PySpark/Scala）。
- 熟悉 Oracle 和 PostgreSQL 数据库，包括编写高效的 SQL 查询。
- 精通在 Azure 上构建 ETL/ELT 工作流和数据编排。
- 有 Azure 存储解决方案（Blob Storage、ADLS Gen2）的经验。
- 对数据建模、数据仓库概念和性能优化有深入理解。
- 熟悉 Azure DevOps 中的 CI/CD 流水线者优先。
- 具备优秀的解决问题能力，能够独立工作或在团队中协作。
最初发布于 Himalayas
查看英文原文
Must Have: Certification: Required – Microsoft Certified: Azure Data Engineer Associate (or higher), Azure Data Factory,
Job Summary:
We are seeking a highly skilled Data Engineer with strong expertise in Azure Data Factory, Azure Databricks, Oracle, and PostgreSQL. The ideal candidate will design and implement robust data pipelines, optimize data flows, and contribute to our cloud data platform initiatives using the Microsoft Azure ecosystem.
Key Responsibilities:
- Design, build, and maintain scalable data pipelines using Azure Data Factory and Databricks (Spark-based).
- Ingest, transform, and orchestrate data from various sources including Oracle and PostgreSQL databases.
- Collaborate with data analysts, data scientists, and other stakeholders to understand data requirements and deliver clean, reliable data.
- Optimize performance of data workflows, especially within Azure environments.
- Implement data quality checks, lineage tracking, and monitoring processes.
- Follow and contribute to data governance, security, and compliance policies.
- Participate in code reviews, documentation, and continuous improvement initiatives.
Required Skills & Experience:
- Strong hands-on experience with Azure Data Factory and Azure Databricks (PySpark/Scala preferred).
- Solid working knowledge of Oracle and PostgreSQL databases, including writing efficient SQL queries.
- Proficient in building ETL/ELT workflows and data orchestration on Azure.
- Experience with Azure storage solutions (Blob Storage, ADLS Gen2).
- Strong understanding of data modeling, warehousing concepts, and performance optimization.
- Familiarity with CI/CD pipelines in Azure DevOps is a plus.
- Excellent problem-solving skills and ability to work independently or in a team.
Originally posted on Himalayas