Data Engineer
在 Digital,我们不仅提供技术，而是提供未来！我们是探索者，知识猎人，科技士，解决问题和改变游戏的人，他们希望获得启发和灵感。我们的 digital 人员总是向前迈出一步：与顶级技术合作，创新先于市场趋势，分享寻找更好的方法的热情。作为一个以人类为中心的组织，我们的团队建立在相互尊重和开放的沟通上，让每个人都能够真实地表达想法并释放自己的潜力。我们自豪的是我们的 digital 聪明头脑，永远不会停止发展他们的技能，继续推进边界，做我们喜欢的事情。果在全球范围内运营，由8个国家的16个地点1000名专家组成。我们正在寻找一个高技能和激励的数据工程师加入我们的动态团队。理想候选人应该拥有丰富的 Azure Databricks,Data Lakehouse 概念，Python,SQL 方面的经验，以及与各种数据来源和平台合作的强有力的背景。这是你的工作：Azure Data bricks: ·设计，开发和维护使用 Azure Data Bricks 的数据管道。·优化并调整 Spark 工作以实现性能和可扩展性。·实施 Spark DataFrames 和 Pandas DataFrams 以有效处理数据。· 实现和管理数据湖泊概念，以有效地存储和处理数据。· 与达尔塔表和达尔塔现场表合作维护版本和交易数据.Azure Data Factory: · 使用 Azure Datafactory 设计和实施数据工作流程。· 从各种源头来处理数据集成，无论是在本地还是云中。数据来源：· 将来自 Azure Data Lake,Azure SQL,Azure Synapse,API 等相关资源的数据集成。·复制活动，绘图数据流量以及处理各种数据格式方面的专业知识。API 集成：· 做 API 调用和处理 JSON 响应的经验。· 处理页面化和复杂的 API 交互的能力.Azure 分析服务 (SSAS): · 用 Azure SSAS 创建和管理表格模型的经验.Azure Key Vault 和数据湖概念：· 对安全存储敏感信息的 Azure Key Valve 的熟悉性。· 了解 Azure Data Lake 的概念和最佳实践。额外技能：· 应对者应该掌握 SQL,并理解有效处理数据的功能和存储程序的使用。他
查看英文原文
At DIGITALL we dont just deliver technology we deliver the future! We are explorers, knowledge-hunters, tech geeks, problem solvers and game changers who want to inspire and be inspired. Our DIGITALL people are always one step forward: working with top-notch technologies, creating innovations ahead of the market trends, sharing the passion for discovering better ways. As a human-centric organization, our teams are built on mutual respect and open communication, allowing everyone to be authentic, express ideas and unleash their potential. We are proud of our DIGITALL bright minds and never stop developing their skills to keep pushing boundaries together and do what we love. DIGITALL operates globally with a team of 1000 experts in 16 locations across 8 countries.
We are seeking a highly skilled and motivated Data Engineer to join our dynamic team. The ideal candidate should have extensive experience in Azure Databricks, Data Lakehouse concepts, Python, SQL, and a strong background in working with various data sources and platforms.
This is your job:
Azure Databricks:
- Design, develop, and maintain data pipelines using Azure Databricks.
- Optimize and tune Spark jobs for performance and scalability.
- Implement Spark DataFrames and Pandas DataFrames for efficient data processing.
Data Lakehouse Concept:
- Implement and manage data lakehouse concepts for effective data storage and processing.
- Work with Delta Tables and Delta Live Tables to maintain versioned and transactional data.
Azure Data Factory:
- Design and implement data workflows using Azure Data Factory.
- Handle data integration from various sources, both on-premises and in the cloud.
Data Sources:
- Integrate data from diverse sources such as Azure Data Lake, Azure SQL, Azure Synapse, APIs, and other relevant sources.
- Expertise in Copy Activity, Mapping Data Flow, and handling various data formats.
API Integration:
- Experience with making API calls and processing JSON responses.
- Ability to handle pagination and complex API interactions.
Azure Analysis Services (SSAS):
· Experience with Azure SSAS for creating and managing tabular models.
Azure Key Vault and Data Lake Concepts:
- Familiarity with Azure Key Vault for secure storage of sensitive information.
- Knowledge of Azure Data Lake concepts and best practices.
Additional Skills:
- The candidate should be proficient in SQL and understand the usage of functions and stored procedures for efficient data handling. He or she should possess a deep understanding of database design and normalization and have experience in database and query performance tuning and optimization.
- Familiarity with dbt (Data Build Tool) for analytics engineering (optional but advantageous).
Your qualifications:
- Bachelor's or higher degree in Computer Science, Data Science, or related field.
- Proven experience of 7+ years as a Data Engineer with a focus on Azure technologies.
- Strong proficiency in Python, SQL, and Spark DataFrames.
- Excellent understanding of data integration, data processing, and data modeling concepts.
- Experience with cloud and on-premises data sources and platforms.
Organizational information:
All applications will be treated in strict confidentiality.
Please note that only shortlisted candidates will be invited to an interview.
Originally posted on Himalayas