AWS DataHub开发工程师
AWS DataHub Developer
职位描述
我们的客户正在寻找一名高级AWS DataHub开发人员，负责在AWS上设计和构建实时、事件驱动的数据服务。该职位以开发人员为中心（应用端），而非以基础设施为导向。您将构建基于Kafka的流处理管道和无服务器数据应用，实现大规模的数据采集、转换和提供。
您将与架构师、数据工程师和产品团队合作，交付安全、可靠、可观察且高度可扩展的解决方案，支持企业级分析和事件驱动的应用程序。
您将负责以下工作（主要职责）
- 设计与交付事件驱动管道：使用AWS Lambda、Step Functions、EventBridge、SNS、SQS、API Gateway构建无服务器数据流。
- 实时流处理：开发Kafka（Apache Kafka/Amazon MSK）消费者/生产者，用于高吞吐、低延迟的流处理和解耦微服务。
- 微服务与API：构建并优化TypeScript（优先）或Python服务/API，用于数据采集、转换和交付。
- AWS数据服务集成：与S3、DynamoDB、Glue、Athena、CloudWatch协作，实现存储、元数据、查询和可观测性。
- 质量与可靠性：在适当的地方实现幂等性、重试、死信队列、精确一次/至少一次语义，以及模式演进策略。
- CI/CD与测试：使用基于Git的工作流程和CI/CD（如GitHub Actions、Jenkins）进行自动化测试（单元/集成/负载测试）和基础设施部署。
- IaC（开发者视角）：使用AWS CDK、Terraform或CloudFormation定义应用层基础设施，强调开发者生产力和可重复性。
- 敏捷协作：参与技术设计、故事估算、同行评审和持续改进。
理想的候选人背景
· 您是一位云原生、应用端开发者，习惯于以事件、流和服 务的方式思考，而非服务器。您注重弹性、可观测性和扩展性，并且能够熟练地将Kafka与AWS无服务器技术结合，以实现业务成果。
查看英文原文
About the Role
Our client is seeking a Senior AWS DataHub Developer to design and build real-time, event-driven data services on AWS. This role is developer-first (application-side) rather than infrastructure-led. You will architect and deliver Kafka-based streaming pipelines and serverless data applications that ingest, transform, and serve data at scale.
You’ll collaborate with architects, data engineers, and product teams to deliver secure, resilient, observable, and highly scalable solutions that power enterprise-grade analytics and event-driven applications.
What You’ll Do (Key Responsibilities)
- Design & Deliver Event-Driven Pipelines: Build serverless data flows using AWS Lambda, Step Functions, EventBridge, SNS, SQS, API Gateway.
- Real-Time Streaming: Develop Kafka (Apache Kafka/Amazon MSK) consumers/producers for high-throughput, low-latency streaming and decoupled microservices.
- Microservices & APIs: Build and optimize TypeScript (preferred) or Python services/APIs for data ingestion, transformation, and delivery.
- AWS Data Services Integration: Work with S3, DynamoDB, Glue, Athena, CloudWatch for storage, metadata, querying, and observability.
- Quality & Reliability: Implement idempotency, retries, dead-letter queues, exactly-once/at-least-once semantics where appropriate, and schema evolution strategies.
- CI/CD & Testing: Use Git-based workflows and CI/CD (e.g., GitHub Actions, Jenkins) with automated tests (unit/integration/load) and infrastructure deployments.
- IaC (Developer View): Define application-layer infrastructure using AWS CDK, Terraform, or CloudFormation—with strong emphasis on developer productivity and repeatability.
- Agile Collaboration: Contribute to technical design, story sizing, peer reviews, and continuous improvement.
Ideal Candidate Profile
· You are a cloud-native, application-side developer who thinks in events, streams, and services—not servers. You design for resiliency, observability, and scale, and you’re comfortable pairing Kafka with AWS serverless to deliver business outcomes.
Originally posted on Himalayas