高级数据工程师 (GCP)
Senior Data Engineer (GCP)
开发工程未标注地域
公司CommIT
薪资未公开
工作地点Armenia
地域资格未标注地域
时区要求日间重叠约 5 小时,基本正常作息
用工类型Full Time
发布时间5 天前
数据来源Himalayas
职位描述
我们正在寻找一位经验丰富的高级数据工程师加入不断壮大的数据工程与分析部门。在此职位中,您将为复杂的客户环境设计和开发先进的数据解决方案,重点在于Google云平台和现代GCP数据技术。您将参与整个数据生命周期的工作——从架构和数据建模到管道开发、数据平台、分析和生产部署。
主要职责:
- 在GCP上设计和开发可扩展的数据解决方案。
- 领导客户数据项目的的技术设计和实施。
- 理解业务和技术需求,并将其转化为有效的数据架构。
- 构建和维护ETL/ELT管道、数据湖、湖仓一体和基于云的数据仓库。
- 为批处理和实时工作负载设计数据模型和集成流程。
- 处理结构化、半结构化和非结构化数据。
- 根据性能、可扩展性、安全性和成本要求选择合适的技术。
- 实现数据质量、监控、治理和编排流程。
- 与数据架构师、数据工程师、DevOps团队、分析师和客户利益相关者紧密合作。
- 参与相关分析、AI和ML解决方案的开发。
要求
- 至少5年作为数据工程师的专业经验 – 必须。
- 具有在GCP上开发数据解决方案的实际经验 – 必须。
- 有使用数据可视化工具(如Looker、Power BI、Tableau或QuickSight)的经验。
- 有数据建模、编排、性能优化和大规模数据处理的经验。
- 强大的Python开发技能,包括构建数据管道和ETL/ELT流程。
- 高水平的SQL能力 – 必须。
- 有设计和开发基于云的数据仓库和湖仓一体解决方案的经验。
- 有使用ETL/ELT和转换工具(如dbt、Dataform、Rivery或类似平台)的经验。
- 熟悉CI/CD、Git、基础设施即代码和生产部署实践。
- 强大的分析和解决问题的能力,注重细节。
- 能够独立学习新技术并在多个项目中工作。
- 在以下GCP服务中有丰富经验:BigQuery、Cloud Storage、Dataflow、Dataproc、Cloud Composer、Cloud Run或Cloud Functions
- 英语流利。
查看英文原文
Description
We are looking for an experienced Senior Data Engineer to join a growing Data Engineering and Analytics department. In this role, you will design and develop advanced data solutions for complex customer environments, with a strong focus on Google Cloud Platform and modern GCP data technologies. You will work across the full data lifecycle- from architecture and data modeling to pipeline development, data platforms, analytics, and production deployment.
Key Responsibilities:
- Design and develop scalable data solutions on GCP.
- Lead the technical design and implementation of customer data projects.
- Understand business and technical requirements and translate them into effective data architectures.
- Build and maintain ETL/ELT pipelines, Data Lakes, Lakehouses, and cloud-based Data Warehouses.
- Design data models and integration processes for Batch and real-time workloads.
- Work with structured, semi-structured, and unstructured data.
- Select the appropriate technologies based on performance, scalability, security, and cost requirements.
- Implement data quality, monitoring, governance, and orchestration processes.
- Work closely with Data Architects, Data Engineers, DevOps teams, analysts, and customer stakeholders.
- Participate in the development of analytics, AI, and ML solutions where relevant.
Requirements
Requirements:
- At least 5 years of professional experience as a Data Engineer – mandatory.
- Proven hands-on experience developing data solutions on GCP – mandatory.
- Experience with data visualization tools such as Looker, Power BI, Tableau, or QuickSight.
- Experience with data modeling, orchestration, performance optimization, and large-scale data processing.
- Strong Python development skills, including building data pipelines and ETL/ELT processes.
- High proficiency in SQL – mandatory.
- Experience designing and developing cloud-based Data Warehouses and Lakehouse solutions.
- Experience with ETL/ELT and transformation tools such as dbt, Dataform, Rivery, or similar platforms.
- Familiarity with CI/CD, Git, Infrastructure as Code, and production deployment practices.
- Strong analytical and problem-solving skills with excellent attention to detail.
- Ability to learn new technologies independently and work across multiple projects.
- Strong experience with several of the following GCP services: BigQuery, Cloud Storage, Dataflow, Dataproc, Cloud Composer, Cloud Run or Cloud Functions
- Fluent English.
Advantages
- Hands-on experience with AWS or Microsoft Azure data services.
- Experience with services such as AWS Glue, Redshift, EMR, Kinesis, Azure Data Factory, Synapse, or Databricks.
- Experience with real-time data processing and streaming architectures.
- Experience with Kafka or other event-driven platforms.
- Knowledge of AI and ML services such as Vertex AI, Gemini, BigQuery ML, SageMaker, Bedrock, or Azure Machine Learning.
- Relevant GCP professional certifications.
- Previous experience working in consulting or customer-facing technology projects.
Originally posted on Himalayas
本页面信息整理自 Himalayas,版权归原发布方所有。职位可能随时关闭,投递请以原始页面为准。
本站只做信息聚合展示,不参与招聘流程,也不向求职者收取任何费用。
该公司其他在招职位
中级数据科学 / AI工程师
AI开发工程限定地区(需当地身份)日间重叠约 2 小时,需偶尔早起或晚睡
技术支持工程师
开发工程职能支持限定地区(需当地身份)
资深DFT工程师
开发工程限定地区(需当地身份)日间重叠约 2 小时,需偶尔早起或晚睡
高级后端工程师
开发工程限定地区(需当地身份)与中国几乎无重叠,需长期倒时差
高级数据库管理员(PostgreSQL和MySQL重点)
开发工程未标注地域