远程工作雷达

高级数据平台工程师 — Kafka & PostgreSQL

Senior Data Platform Engineer — Kafka & PostgreSQL

开发工程限定地区（需当地身份）
公司Mirantis
薪资未公开
工作地点United States
地域资格限定地区（需当地身份）
时区要求日间重叠约 9 小时，基本正常作息
用工类型Full Time
发布时间今天
数据来源Himalayas
前往 Himalayas 查看并投递 →
注意地域限制：该职位明确限定在 United States 招聘。如果你是位于中国大陆的求职者，通常需要当地工作身份才能投递，或需与雇主确认是否接受独立合同（Contractor）形式合作。

我们正在寻找一位经验丰富的高级数据平台工程师，负责管理驱动 k0rdent-ai 平台——企业 GPU 基础设施的多租户控制平面——的事件流、事务数据库和定制连接基础架构。每台集群的部署、每个小时 GPU 的使用以及每个租户操作都会产生高基数的元数据，这些数据必须进行事务性记录并安全隔离。
你将设计核心数据管道，将我们的流数据平面（通过 Strimzi 使用 Apache Kafka）连接到我们的关系型数据层（通过 CloudNativePG 使用开源 PostgreSQL），该数据层运行在弹性 Kubernetes 集群和高性能裸金属硬件上。除了标准运维外，该职位的重点还包括定制生态系统工程：深入的 Go（Golang）系统编程，用于构建定制的开源客户端和连接器，设计专用集成库，并开发专有的中间件，以将我们的数据库多租户功能与企业身份平面（如 Active Directory (AD)/LDAP）连接起来。
由于该系统是平台上每个租户操作的事务性记录，该职位还负责其灾难恢复策略——设计平台在整区域或集群丢失时的生存能力，而不仅仅是单节点故障。

主要职责

  • 混合数据库架构：设计、部署和运营高可用的开源 PostgreSQL 拓扑结构（通过 Kubernetes 上的 CloudNativePG 和裸金属上的原生 Patroni 风格拓扑结构），以及分布式 Apache Kafka 集群（通过 Strimzi），覆盖容器化的 Kubernetes 环境和高性能裸金属硬件。
  • 定制客户端与连接器开发：使用 Go（Golang）从零开始编写企业级开源 Kafka 客户端、独立连接器服务和实用程序库，以扩展数据能力，当现成工具无法满足需求时。
  • 企业身份与数据集成：设计并构建系统集成，将 PostgreSQL 身份验证和行级安全（RLS）策略评估连接到企业 Active Directory (AD)、LDAP 和 OIDC 身份提供者——通过角色映射和会话上下文层（例如映射的 Postgres 角色、RLS 策略使用的 JWT 声明）而非直接连接。
  • 事务架构与 CDC：通过 Debezium 和 Kafka Connect 扩展高吞吐量变更数据捕获（CDC）管道。实现具有弹性的架构模式，以确保绝对的数据一致性与可靠性。
查看英文原文

We are looking for an experienced Senior Data Platform Engineer to own the event-streaming, transactional database, and custom connectivity backbone driving the k0rdent-ai platform — our multi-tenant control plane for enterprise GPU infrastructure. Every cluster provisioned, every GPU-hour consumed, and every tenant action produces high-cardinality metadata that must be transactionally recorded and securely isolated.
You will architect the core pipeline connecting our streaming data plane (Apache Kafka via Strimzi) to our relational data tier (Open Source PostgreSQL via CloudNativePG), which runs across both elastic Kubernetes clusters and high-performance bare metal hardware. Beyond standard administration, a substantial focus of this role is custom ecosystem engineering: deep Go (Golang) systems programming to build custom open-source clients and connectors, design specialized integration libraries, and develop proprietary middleware that bridges our database multi-tenancy with enterprise identity planes like Active Directory (AD)/LDAP.
Because this system is the transactional record of every tenant action across the platform, this role also owns its disaster-recovery posture — designing how the platform survives a full region or cluster loss, not just a single-node failure.
Main Responsibilities

  • Hybrid Database Architecture: Design, deploy, and operate high-availability Open Source PostgreSQL topologies (via CloudNativePG on Kubernetes and native Patroni-style topologies on bare metal) and distributed Apache Kafka clusters (via Strimzi), across both containerized Kubernetes environments and high-performance bare metal hardware.
  • Custom Client & Connector Development: Write custom, enterprise-grade open-source Kafka clients, standalone connector services, and utility libraries from scratch using preferably Go (Golang) to extend data capabilities where off-the-shelf tooling falls short.
  • Enterprise Identity & Data Integration: Architect and build system integrations connecting PostgreSQL authentication and row-level security (RLS) policy evaluation to enterprise Active Directory (AD), LDAP, and OIDC identity providers — via role-mapping and session-context layers (e.g., mapped Postgres roles, JWT claims consumed by RLS policies) rather than a direct connection.
  • Transactional Architecture & CDC: Scale high-throughput Change Data Capture (CDC) pipelines via Debezium and Kafka Connect. Implement resilient architectural patterns to maintain absolute data integrity between databases and topics without dual-write risk.
  • Cross-Region Resilience & Disaster Recovery: Design and operate cross-region failover and disaster-recovery orchestration for both PostgreSQL and Kafka — including replication topology (sync vs. async trade-offs), split-brain prevention via quorum/witness mechanisms, and explicit RPO/RTO targets for a full region or cluster loss, not just single-node HA.
  • PostgreSQL & Infra Tuning: Optimize PostgreSQL instances for heavy ingestion and zero-downtime operations. Tune Write-Ahead Logs (WAL), logical replication streams, connection pooling (PgBouncer), and configure underlying Kubernetes infrastructure primitives (CSI storage volumes and CNI network paths) to eliminate replication lag and unnecessary cross-node latency.
  • GitOps & Self-Service Platforming: Maintain a strictly declarative infrastructure-as-code (IaC) culture using Terraform and ArgoCD, creating self-service workflows so internal product teams can securely provision databases, topics, schemas, and ACLs through code.
  • Security & Isolation: Implement strict multi-tenant isolation, combining database-level row-level security (RLS) with CNI network policies, mTLS, and Kafka topic-level RBAC.

We don't expect any one candidate to check every box below — if your experience is strong across most of these areas, we encourage you to apply.
Must Have
Platform Seniority: 10+ years in systems, software, data, or platform engineering, with a track record of owning large-scale, business-critical data infrastructure.

  • Programming: Go (Golang) or similar engineering skills — comfortable writing clean, concurrent systems code, building custom client libraries, and interacting directly with database/network APIs.
  • Open Source PostgreSQL Expertise: 5+ years managing native PostgreSQL. Deep knowledge of database internals (WAL streams, logical replication, publications/subscriptions, and connection architectures) across both Kubernetes stateful environments and bare metal physical nodes.
  • Production PostgreSQL-on-Kubernetes: Hands-on experience running PostgreSQL on Kubernetes via a mature Postgres operator — CloudNativePG (CNPG) preferred, but equivalent experience with Zalando's postgres-operator or a comparable operator is acceptable. What matters is depth with declarative cluster lifecycle, failover, and backup/PITR management, not the specific product name.
  • Production Kafka: Strong experience managing Apache Kafka at scale on Kubernetes via a declarative operator — Strimzi preferred, but equivalent experience with Confluent for Kubernetes, Koperator, or a comparable operator is acceptable. What matters is depth with operator-managed broker lifecycle and scaling, not the specific product name.
  • Custom Data Movement & CDC: Proven experience tuning Kafka Connect and Debezium pipelines. Experience writing or contributing to open-source database connectors, Kafka client libraries, or integration frameworks.
  • Enterprise Identity Planes: Hands-on experience building integrations connecting distributed systems to an enterprise IAM/SSO provider — Active Directory, LDAP, Okta, Azure AD, or a generic OIDC provider all count; the specific product matters less than real experience integrating application-level authorization with an enterprise identity system.
  • Kubernetes & Cloud Infrastructure: Strong grasp of K8s primitives (Pod lifecycles, Operators, CSI storage layers, and CNI overlay routing), backed by production experience in AWS or GCP.

Nice to Have

  • Experience designing and operating cross-region disaster recovery and failover orchestration for stateful systems (Postgres and/or Kafka), including split-brain prevention and RPO/RTO trade-off decisions.
  • Active contributions to open-source Kafka connectors, PostgreSQL operators (e.g., CloudNativePG, Zalando), or Go-based database libraries.
  • Experience with schema governance for event streams (e.g., Karapace or Confluent Schema Registry) and compatibility-mode management for evolving event contracts.
  • Familiarity with high-cardinality telemetry ingestion using Apache Flink or Kafka Streams.
  • Familiarity with US enterprise compliance benchmarks (SOC 2, ISO 27001).
  • Hands-on knowledge of open-source Redis (caching, pub/sub, or as a lightweight data store) — a huge plus.
  • Hands-on knowledge of open-source MongoDB (document modeling, replication, sharding) — a huge plus.

Education and Experience
Bachelor's degree in Computer Science & Engineering (or a related field), or 10+ years of equivalent related experience.
What does Mirantis offer you?

  • Work with an established Silicon Valley leader in the cloud infrastructure industry;
  • Work with exceptionally passionate, talented and engaging colleagues, helping Fortune 500 and Global 2000 customers implement next-generation cloud technologies;
  • Be a part of cutting-edge, open-source innovation;
  • Thrive in the high-energy environment of a young company where openness, collaboration, risk-taking, and continuous growth are valued;
  • Professional development and training;
  • Attend conferences and working groups;
  • Company outings, happy hours, hackathons, and tech talks;
  • Receive a competitive compensation package with a strong benefits plan.

We are a Leader for Container Management in G2 (#2 after AWS)!
Mirantis, an IREN company, is the Kubernetes-native AI infrastructure company, enabling organizations to build and operate scalable, secure, and sovereign infrastructure for modern AI, machine learning, and data-intensive applications. By combining open source innovation with deep expertise in Kubernetes orchestration, Mirantis empowers platform engineering teams to deliver composable, production-ready developer platforms across any environment—on-premises, in the cloud, at the edge, or in sovereign data centers. As enterprises navigate the growing complexity of AI-driven workloads, Mirantis delivers the automation, GPU orchestration, and policy-driven control needed to manage infrastructure with confidence and agility. Committed to open standards and freedom from lock-in, Mirantis ensures that customers retain full control of their infrastructure strategy.
Originally posted on Himalayas

本页面信息整理自 Himalayas，版权归原发布方所有。职位可能随时关闭，投递请以原始页面为准。 本站只做信息聚合展示，不参与招聘流程，也不向求职者收取任何费用。

该公司其他在招职位

← 返回全部职位