高级数据工程师

We are seeking a highly skilled Senior Data Engineer to join our team and play a pivotal role in designing, implementing, and managing new data technology infrastructure. You will be responsible for building and maintaining a scalable and robust cloud-based platform that consolidates existing legacy data services. The platform will enable efficient data management, automation of processes, and provide high-quality data services to our users and tooling.The role is 100% remote.

Key Responsibilities

• Design, develop, and implement data pipelines and ETL processes to ingest, transform, and load data into the data platform.

• Collaborate with the Lead Architect and Cloud Engineer to ensure the seamless integration of data services within the cloud infrastructure.

• Develop data models and schemas to support efficient data storage and retrieval.

• Implement data quality and validation processes to ensure the accuracy and consistency of data.

• Optimize data processes and queries for performance and scalability.

• Collaborate with business stakeholders to understand their data requirements and provide insights and solutions for data-driven decision-making.

• Work closely with data scientists to provide them with the necessary data and tooling for developing revenue-generating insights and models.

• Implement data governance and security controls to ensure data privacy and compliance with regulations.

• Stay updated with the latest data technologies and trends and make recommendations for technology adoption and improvements.

Skills, Knowledge and Expertise

• Excellent understanding of Azure data tools, especially Databricks

• Proven experience as a Data Engineer, working with Azure data services.

• Strong understanding of data modeling concepts and relational and non-relational databases.

• Proficiency in SQL and experience with data integration and ETL tools (e.g., Azure Data Factory, Azure Databricks).

• Experience with data warehousing concepts and technologies (e.g., Azure Synapse Analytics, Azure Data Lake Storage).

• Familiarity with big data technologies and frameworks (e.g., Hadoop, Spark) is a plus.

• Knowledge of data governance and data security best practices.

• Strong problem-solving skills and attention to detail.

• Excellent communication and collaboration skills to work effectively within a team environment.

• Experience with Agile methodologies is a plus.

• Comfortable problem solving alone as well as part of a team.

• Confident reaching out to stakeholders and colleagues in order to solve problems and understand issues.

• Happy in a remote working environment.

• Keen interest in new technologies and learning about them.

Benefits

• Competitive salary and compensation package.

• Exclusive access to AWS & Databricks certifications, training, and partner tools.

• Remote work with regular team socials.

• Work with a highly experienced, motivated team of experts in Cloud, Big Data and Machine Learning.

‍

Originally posted on Himalayas