Azure数据工程师 - 自由职业
Azure Data Engineer - freelance
通过多元化、公平和包容实现增长。作为一家有道德的企业，我们坚持做正确的事——包括确保平等的机会，并为每个人打造一个安全、受尊重的工作环境。我们相信多元化能推动个人和业务的成长。我们致力于构建一个包容的社区，无论背景、身份或其他个人特征如何，所有员工都能蓬勃发展。
我们正在寻找具有 Databricks 经验的 Azure 数据工程师。理想的候选人应具备设计和实施数据管道、数据仓库解决方案和 ETL 工作流的实际经验。
职责
- 在 Azure 平台上设计和实现数据处理系统。这包括编写高效且可扩展的代码，以处理、转换和清理大量结构化和非结构化数据。
- 构建数据管道，从各种来源（如数据库、API 或流媒体平台）获取数据。
- 整合和转换数据，以确保其与目标数据模型或格式兼容。
- 具备设计和优化数据存储（数据湖、数据仓库、分布式文件系统）的经验。
- 熟悉数据存储和检索的分区、压缩和优化。
- 识别并解决瓶颈，优化查询，并实施缓存策略，以提高数据检索速度和整体系统效率。
- 与跨职能团队（包括数据科学家、分析师和业务利益相关者）合作，了解他们的需求并提供技术解决方案。
- 对解决方案交付过程负责并表现出责任感，确保所有任务高效、有效地完成，并达到最高标准。
- 能够在敏捷方法下工作。
要求
- 具备 Azure 云基础设施和 Databricks 实施经验。
- 具备 SQL 经验。
- 具备 Python 和 PySpark 经验。
- 具备良好的沟通能力，能够向同事和业务利益相关者清晰具体地传达信息。
- 具备敏捷方法经验——支持工具（JIRA、Azure DevOps）。
- 英语至少达到 B2 级，最好是 C1 级。
加分项：
- 具备其他云技术、数据仓库、数据治理和业务分析的经验或熟悉程度。
如果缺少其中一两项资格？我们仍然希望收到你的申请！如果你有积极的心态，我们会提供支持。
查看英文原文
Growth through diversity, equity, and inclusion. As an ethical business, we do what is right — including ensuring equal opportunities and fostering a safe, respectful workplace for each of us. We believe diversity fuels both personal and business growth. We're committed to building an inclusive community where all our people thrive regardless of their backgrounds, identities, or other personal characteristics.
We are seeking Azure Data Engineers experienced with Databricks. Ideal candidates will have hands-on experience in designing and implementing data pipelines, data warehousing solutions, and ETL workflows.
Tasks
- Designing and implementing data processing systems on Azure platform. This involves writing efficient and scalable code to process, transform, and clean large volumes of structured and unstructured data.
- Building data pipelines to ingest data from various sources such as databases, APIs, or streaming platforms.
- Integrating and transforming data to ensure its compatibility with the target data model or format.
- Experience in designing and optimizing of data storage (data lakes, data warehouses, distributed file systems)
- Familiarity with partitioning, compression, optimization of data storage and retrieval.
- Identifying and resolving bottlenecks, tuning queries, and implementing caching strategies to enhance data retrieval speed and overall system efficiency.
- Collaborating with cross-functional teams including data scientists, analysts, and business stakeholders to understand their requirements and provide technical solutions.
- Taking ownership and demonstrating responsibility throughout the solution delivery process, ensuring that all tasks are executed efficiently, effectively, and to the highest standard.
- Ability to work under Agile methodologies.
Requirements
- Experience in Azure cloud-based infrastructure and Databricks implementation.
- Experience with SQL.
- Experience with Python, PySpark.
- Very good level of communication including ability to convey information clearly and specifically to co-workers and business stakeholders.
- Working experience in Agile methodologies – supporting tools (JIRA, Azure DevOps).
- English at least at B2 level, ideally C1.
Nice to have:
- Experience or familiarity with other cloud technologies, data warehouses, data governance, and business analysis.
Missing one or two of these qualifications? We still want to hear from you! If you bring a positive mindset, we'll provide an environment where you feel valued and empowered to learn and grow.
Originally posted on Himalayas