自由职业 高级 Databricks 平台工程师
Freelance Senior Databricks Platform Engineer
adesso Belgium，我们是谁？我们在 adesso 旗下运营，是一家 IT（初创）服务提供商，在多个行业拥有成功的业绩。adesso 的比利时新实体是 adesso 集团扩张和增长战略的一部分。
要求
开始时间：尽快- 远程工作
时长：6 个月
薪资：待定- 根据您的所在地而定
我们正在为一位国际客户紧急寻找一名高级 Databricks 平台工程师。
我们目前寻找的职位更接近 Databricks 平台工程师，而不是传统的数据工程师或 BI 咨询顾问。目标是帮助设计和构建基于 Databricks 的数据产品平台，重点在于平台架构、治理、基础设施即代码、持续集成/持续交付、安全，以及让多个团队能够构建和运营数据产品。
您在项目中的职责：
设计与构建数据平台
- 在 Databricks 上设计和实现 Lakehouse 架构（银币/青铜-白银-黄金模式、Delta Lake、Unity Catalog）。
- 使用 PySpark 和 Spark SQL 构建稳健的批处理和流数据管道，通过 Databricks Workflows / Jobs 进行编排。
- 为分析用例建模数据（例如用于 BI 的星型模式、用于数据科学的整理数据集）。
数据采集、转换与治理
- 从 API、文件、数据库和 SaaS 源采集数据到数据湖和 Lakehouse 中。
- 实现清晰且可测试的转换逻辑（数据质量检查、去重、SCD、CDC、业务规则）。
- 应用数据治理最佳实践（目录管理、数据血缘、基于角色的访问控制、敏感数据屏蔽）。
运维与优化
- 为 Databricks 资产（笔记本、作业、管道、工作流）设置 CI/CD 和部署管道（Azure DevOps / GitHub）。
- 使用 Terraform 自动化 IaC 设置和复用。
- 监控、优化和排查管道和集群的性能、健壮性和成本效率。
- 协助定义和实施团队内的最佳实践、编码标准和可复用组件。
- 您的简历 - 必须的技术技能与知识：
- 有以下方面的实际经验：
- Databricks 平台架构
- Unity Catalog 和治理
- Databricks 的 CI/CD 和 DevOps - 企业 GitHub
- Terraform / 基础设施即代码
- Lakehouse 架构
- 数据产品平台赋能
- 了解知识图谱、本体和代理 AI 是一大优势。
- Unity 目录、CI/CD、IAC 和 Terraform 技能在模式实现和集成方面具备能力
- 为什么加入 adesso Belgium？
- 工作
查看英文原文
adesso Belgium, who are we? We are an IT (start-up) provider working under the adesso umbrella with proven track record in different industries. The new Belgian entity of adesso is part of the expansion and growth strategy of the adesso group.
Requirements
Start date: ASAP- Remote work
Duration: 6 months
Rate: TBD- depending on your location
For one of our international customers, we are urgently looking for a senior Databricks Platform engineer.
The role we are currently looking for is closer to a Databricks Platform Engineer than a traditional Data Engineer or BI consultant. The objective is to help design and build a Data Product Platform on Databricks, with a strong focus on platform architecture, governance, Infrastructure as Code, CI/CD, security, and enabling multiple teams to build and operate data products.
Your Role in projects:
Design & build data platforms
- Design and implement Lakehouse architectures on Databricks (medallion / bronze‑silver‑gold patterns, Delta Lake, Unity Catalog).
- Build robust batch and streaming data pipelines using PySpark and Spark SQL, orchestrated via Databricks Workflows / Jobs.
- Model data for analytics use cases (e.g. star schema for BI, curated datasets for data science).
Ingest, transform & govern data
- Ingest data from APIs, files, databases and SaaS sources into data lakes and lakehouses.
- Implement clear and testable transformation logic (data quality checks, deduplication, SCD, CDC, business rules).
- Apply data governance best practices (cataloguing, lineage, RBAC, masking of sensitive data).
Operate & optimize
- Set up CI/CD and deployment pipelines (Azure DevOps / GitHub) for Databricks assets (notebooks, jobs, pipelines, workflows).
- Using Terraform to automated IaaC setup and reusability
- Monitor, optimize and troubleshoot pipelines and clusters for performance, robustness and cost efficiency.
- Help define and implement best practices, coding standards and reusable components within the team.
Your profile - Required technical skills & knowledge:
Proven experience in:
- Databricks platform architecture
- Unity Catalog and governance
- CI/CD and DevOps for Databricks - Enterprise GitHub
- Terraform / Infrastructure as Code
- Lakehouse architecture
- Data Product Platform enablement
- The Knowledge Graph, ontology and agentic AI are considered a strong plus.
- Unity catalogue, CICD, IAC and terraform skills in mode implementation and integration
Why join adesso Belgium?
- Work in an international group with strong local expertise in Belgium.
- Collaborate with cross-functional teams within the international data & AI adesso community.
- Flexible freelance setup with the support of a growing adesso Belgium community.
Originally posted on Himalayas