云数据工程师 — Snowflake BigQuery 迁移
Cloud Data Engineer — Snowflake BigQuery Migration
职位概述
我们正在寻找一位经验丰富的高级数据工程师/数据迁移架构师,负责在Google Cloud上领导大规模企业数据平台现代化项目。理想的候选人应具备将大规模数据仓库(数百TB)迁移到BigQuery的经验,设计高性能数据架构,并交付生产级分析平台。具有从Snowflake迁移到BigQuery的经验者优先;其他平台迁移经验的候选人需通过其他项目展示对Snowflake的深入理解。
主要职责
数据平台现代化与迁移
- 领导大规模数据仓库迁移到BigQuery的评估、设计和执行,包括数据量超过数百TB的环境。
- 定义迁移策略、数据模型、治理框架和性能优化方法。
- 推动现代化项目,提升可扩展性、可靠性、成本效率和运营卓越性。
- 与相关方协作,确保迁移过程顺畅,最大程度减少业务中断。
数据工程与架构
- 设计、开发和维护支持批处理和流式处理任务的高性能数据管道。
- 在Google Cloud Platform上构建安全、可扩展且高可用的数据湖和数据仓库解决方案。
- 建立企业级数据处理框架、集成模式和治理标准。
- 确保数据质量、可观测性、血缘关系和数据生态系统的运营可靠性。
BigQuery工程与优化
- 担任BigQuery架构、性能优化和最佳实践的技术专家。
- 实施高级SQL优化技术,提升查询性能并降低运营成本。
- 设计高效的分区、聚类、存储和工作负载管理策略。
- 优化数据处理管道,提升性能、可扩展性和成本效率。
云架构与交付领导力
- 设计并实现符合业务和技术需求的企业级云架构。
- 从方案设计到实施和生产部署,领导面向客户的项目。
- 为工程团队提供技术领导、指导和架构建议。
- 支持客户研讨会、架构评审和战略路线图讨论。
DevOps
查看英文原文
Role Overview
We are seeking an experienced Senior Data Engineer / Data Migration Architect to lead large-scale enterprise data platform modernization initiatives on Google Cloud. The ideal candidate will have a proven track record of migrating large-scale data warehouses (hundreds of terabytes) to BigQuery, designing high-performance data architectures, and delivering production-grade analytics platforms. Experience migrating from Snowflake to BigQuery is highly preferred; candidates with migration experience from other platforms must demonstrate strong exposure to Snowflake through other engagements.
Key Responsibilities
Data Platform Modernization & Migration
- Lead the assessment, design, and execution of large-scale data warehouse migrations to BigQuery, including environments exceeding hundreds of terabytes of data.
- Define migration strategies, data models, governance frameworks, and performance optimization approaches.
- Drive modernization initiatives that improve scalability, reliability, cost efficiency, and operational excellence.
- Collaborate with stakeholders to ensure seamless migration with minimal business disruption.
Data Engineering & Architecture
- Design, develop, and maintain high-performance data pipelines supporting both batch and streaming workloads.
- Architect secure, scalable, and highly available data lakes and data warehouse solutions on Google Cloud Platform.
- Establish enterprise-grade data processing frameworks, integration patterns, and governance standards.
- Ensure data quality, observability, lineage, and operational reliability across the data ecosystem.
BigQuery Engineering & Optimization
- Serve as the technical expert for BigQuery architecture, performance optimization, and best practices.
- Implement advanced SQL optimization techniques to improve query performance and reduce operational costs.
- Design efficient partitioning, clustering, storage, and workload management strategies.
- Optimize data processing pipelines for performance, scalability, and cost efficiency.
Cloud Architecture & Delivery Leadership
- Design and implement enterprise cloud architectures aligned with business and technical requirements.
- Lead customer-facing projects from solution design through implementation and production deployment.
- Provide technical leadership, mentoring, and architectural guidance to engineering teams.
- Support client workshops, architecture reviews, and strategic roadmap discussions.
DevOps, Automation & Platform Engineering
- Automate infrastructure provisioning and platform deployment using Infrastructure as Code (IaC) practices.
- Design and maintain CI/CD pipelines supporting data platform development and deployment.
- Implement monitoring, alerting, testing, and operational automation capabilities.
- Promote DevOps best practices across development and operations teams.
Cross-Functional Collaboration
- Partner closely with Data Scientists, Data Analysts, Software Engineers, and Business Stakeholders to deliver end-to-end data solutions.
- Support advanced analytics, machine learning, and data-driven decision-making initiatives.
- Translate business requirements into scalable technical solutions and actionable delivery plans.
Mandatory Qualifications
- Minimum 5 years of experience designing, developing, and delivering high-performance data pipelines for both streaming and batch processing workloads.
- Proven experience migrating large-scale enterprise data warehouses to BigQuery, preferably from Snowflake.
- Experience with data warehouse environments measured in hundreds of terabytes.
- Strong expertise in BigQuery architecture, administration, optimization, and operations.
- Demonstrated experience building and maintaining secure, reliable, and scalable data lakes and data warehouses on Google Cloud Platform.
- Proven ability to design enterprise cloud solutions and lead customer projects through successful delivery.
- Hands-on experience with infrastructure automation, DevOps practices, and CI/CD implementation.
- Google Cloud Professional Data Engineer certification.
- Fluent English communication skills, both written and verbal.
Preferred Qualifications
- Fluent German language skills.
- Experience in technical consulting and customer-facing advisory roles.
- Advanced SQL tuning expertise focused on performance optimization and cost efficiency.
- Experience designing and implementing streaming data pipelines with a strong understanding of event delivery semantics and distributed processing concepts.
- Proven experience building production-grade, internet-scale Big Data solutions using managed cloud services.
- Hands-on experience with Snowflake architectures and migration methodologies.
- Experience with Ab Initio is considered an advantage.
- Strong collaboration experience with Data Scientists, Data Analysts, and Software Engineering teams in enterprise environments.
Preferred Technical Expertise
- Google Cloud Platform (GCP)
- BigQuery
- Data Lake and Data Warehouse Architecture
- Snowflake
- Data Migration & Modernization Programs
- Streaming Data Platforms
- Infrastructure as Code (IaC)
- CI/CD and DevOps Tooling
- SQL Performance Tuning
- Data Governance, Security, and Compliance
- Enterprise Data Architecture
- Analytics and Machine Learning Enablement
Originally posted on Himalayas