Data Engineer
角色：数据工程师 （运输重点）高级职位：常规，定期+或高级客户端：一个技术公司构建和优化可扩展的数据网，在其平台上支持跨产品使用案例。客户场地：爱尔兰工作地点：波兰项目描述：开发和优化一个数据网，支持不同产品中的应用案例在客户平台内的。该工作涉及建立和维护大规模的数据处理基础设施和数据转换逻辑。这是一个关键的实践工程作用，专注于构建和優化高度可扩展和可靠的数据网。工程师将建立和支持大规模数据处理基础设施和转型逻辑。技术堆：核心技术：· Apache Spark 需要深入的经验，包括：· 部署，管理和成本优化 Spark 集群 · 写高性能 Spark ETL 工作 ·理想情况下，在大型分析表 （例如 Apache ·冰山）上写体验 · 基础设施作为代码 (IaC): 地形，与 AWS 的经验 · CI/CD:詹金斯 ·测试：单位，整合和端到端测试团队结构：整个项目相当广泛；然而，这个特定团队由6个 FTEs 组成：• 3x 数据工程师 • 1x 数据运营商 • 2x Python 开发人员 工作时间：欧盟时区语言水平：需要优秀的口头和书面沟通技能能够清楚地向项目经理和运营团队传达技术决策。额外信息：•背景检查要求 候选人的关键特征：·特殊分析和解决问题的能力 ·复杂跨服务问题进行调试的能力 ·坚持清洁代码原则 原始发布在喜马拉雅山
查看英文原文
Role: Data Engineer (Transport Focus)
Seniority: Regular, Regular+, or Senior
Client:
A technology company building and optimising a highly scalable Data Mesh that powers cross-product use cases across their platform.
Client Location: Ireland
Work Location: Poland
Project Description:
Development and optimisation of a Data Mesh supporting use cases across various products within the client’s platform. The work involves building and maintaining large-scale data processing infrastructure and data transformation logic.
This is a critical, hands-on engineering role focused on building and optimising ahighly scalable and reliable Data Mesh that powers cross-product use cases across the platform. The engineer will build and support large-scale data processing infrastructure and transformation logic.
Tech Stack:
Core Technologies:
- Apache Spark – deep, proven experience required, including:
- Deployment, management, and cost optimisation of Spark clusters
- Writing high-performance Spark ETL jobs
- Ideally, experience writing to large analytical tables (e.g., Apache
- Iceberg)
- Infrastructure as Code (IaC): Terraform, experience with AWS
- CI/CD: Jenkins
- Testing: Unit, Integration, and End-to-End testing
Team Structure:
The overall project is quite extensive; however, this particular team consists of 6 FTEs:
• 3x Data Engineers
• 1x Data Operator
• 2x Python Developers
Working Hours: EU timezone
Language Level:
Excellent verbal and written communication skills required – ability to clearly communicate technical decisions to Project Managers and Operational Teams.
Additional Information:
• Background check required
Key candidate traits:
- Exceptional analytical and problem-solving skills
- Ability to debug complex cross-service issues
- Strong adherence to Clean Code principles
Originally posted on Himalayas