数据工程师

• Azure Data Engineer with 6+ years of experience designing, developing, and optimizing enterprise-scale data platforms using Microsoft Azure and Snowflake

• Proven expertise in building scalable data pipelines, data warehouses, and analytics solutions leveraging Azure Data Factory (ADF), Azure Databricks, Azure Synapse Analytics, Snowflake, Python, SQL, and Spark

• Experienced in data modeling, ETL/ELT development, cloud migration, performance tuning, data governance, and implementing CI/CD practices

• Strong background in integrating data from multiple sources and enabling business intelligence and advanced analytics solutions

Key Skills:

• Azure Data Factory (ADF)

• Azure Databricks

• Azure Synapse Analytics

• Snowflake Data Cloud

• Python, PySpark, SQL

• Azure Data Lake Storage (ADLS Gen2)

• Azure Functions

• Delta Lake

• Data Warehousing & Data Modeling

• ETL / ELT Development

• CI/CD (Azure DevOps, Git)

• Performance Tuning & Optimization

• Data Governance & Security

• Power BI Integration

• Agile & Scrum Methodologies

What you will do:

• Design, develop, and maintain scalable data engineering solutions on Azure and Snowflake platforms

• Build and optimize ETL/ELT pipelines using Azure Data Factory, Databricks, and Snowflake to ingest, transform, and load data from various structured and unstructured sources

• Develop and maintain enterprise data warehouses, data marts, and analytical datasets supporting reporting, BI, and advanced analytics initiatives
• Implement data transformation and orchestration processes using PySpark, SQL, Python, and Snowflake stored procedures

• Design and manage cloud-based data lakes using Azure Data Lake Storage (ADLS Gen2) and Delta Lake architecture

• Collaborate with business stakeholders, data analysts, and architects to gather requirements and translate them into scalable data solutions

• Monitor and optimize data pipeline performance, ensuring reliability, scalability, and cost efficiency

• Implement data quality frameworks, validation rules, and monitoring processes to ensure accurate and trusted data

• Manage Snowflake objects including databases, schemas, warehouses, tasks, streams, and data-sharing capabilities

• Develop and maintain dimensional and normalized data models following industry best practices

• Implement security controls, access management, encryption, and governance policies across Azure and Snowflake environments

• Support cloud migration and modernization initiatives from on-premises or legacy platforms to Azure and Snowflake

• Automate deployment processes using Azure DevOps, Git repositories, and CI/CD pipelines

• Troubleshoot production issues, perform root cause analysis, and provide timely resolution for data-related challenges

• Participate in Agile ceremonies, code reviews, and technical design discussions to ensure high-quality solution delivery

• Create technical documentation, architecture diagrams, operational runbooks, and knowledge-transfer materials

Preferred Certifications:

• Microsoft Certified: Azure Data Engineer Associate (DP-203)

• SnowPro Core Certification

Originally posted on Himalayas