机器学习工程师 — 推理优化
Machine Learning Engineer — Inference Optimization
职位描述
我们正在寻找一名机器学习工程师，负责推动大规模模型推理性能的边界。你将在研究和生产系统的交汇点工作——将前沿模型转化为快速、可靠且成本高效的系统，为真实用户提供服务。
这个职位适合喜欢深入技术工作的人员，能够对系统进行到内核/GPU级别的分析，并将研究想法转化为生产环境中的性能提升。
你将负责
· 优化大规模ML模型在生产环境中的推理延迟、吞吐量和成本
· 分析和定位GPU/CPU推理流水线的瓶颈（内存、内核、批处理、IO）
· 实现并调优以下技术：
· 量化（fp16、bf16、int8、fp8）
· KV缓存优化与复用
· 推测解码、批处理和流式处理
· 推理用的模型剪枝或架构简化
· 与研究工程师合作，将新的模型架构投入生产
· 构建和维护推理服务系统（如Triton、自定义运行时或定制化架构）
· 在不同硬件（NVIDIA/AMD GPU、CPU）和云环境中进行性能基准测试
· 在真实负载下提升系统的可靠性、可观测性和成本效率
我们寻找的人
· 具有机器学习推理优化或高性能机器学习系统方面的丰富经验
· 对深度学习内部机制有扎实的理解（注意力机制、内存布局、计算图）
· 具有PyTorch（或其他类似框架）和模型部署的实际操作经验
· 熟悉GPU性能调优（CUDA、ROCm、Triton或内核级优化）
· 有为真实用户扩展推理的经验（而不仅仅是研究基准）
· 能够在快节奏的初创环境中工作，具备主人翁意识和应对模糊性的能力
加分项
· 具有LLM或长上下文模型推理的经验
· 了解推理框架（TensorRT、ONNX Runtime、vLLM、Triton）
· 具有跨不同硬件供应商优化的经验
· 在机器学习系统或推理工具链中有开源贡献
· 具有分布式系统或低延迟服务的背景
加入我们为什么
· 对性能关键系统拥有真正的主导权
· 直接影响产品的可靠性和单位经济性
· 与研究、基础设施和产品团队紧密合作
· 竞争性的薪酬+A轮阶段有意义的股权
· 一个重视工程品质而非炒作的团队
最初发布于喜马拉雅斯
查看英文原文
About the Role
We’re looking for a Machine Learning Engineer to own and push the limits of model inference performance at scale. You’ll work at the intersection of research and production—turning cutting-edge models into fast, reliable, and cost-efficient systems that serve real users.
This role is ideal for someone who enjoys deep technical work, profiling systems down to the kernel/GPU level, and translating research ideas into production-grade performance gains.
What You’ll Do
- Optimize inference latency, throughput, and cost for large-scale ML models in production
- Profile and bottleneck GPU/CPU inference pipelines (memory, kernels, batching, IO)
- Implement and tune techniques such as:
- Quantization (fp16, bf16, int8, fp8)
- KV-cache optimization & reuse
- Speculative decoding, batching, and streaming
- Model pruning or architectural simplifications for inference
- Collaborate with research engineers to productionize new model architectures
- Build and maintain inference-serving systems (e.g. Triton, custom runtimes, or bespoke stacks)
- Benchmark performance across hardware (NVIDIA / AMD GPUs, CPUs) and cloud setups
- Improve system reliability, observability, and cost efficiency under real workloads
What We’re Looking For
- Strong experience in ML inference optimization or high-performance ML systems
- Solid understanding of deep learning internals (attention, memory layout, compute graphs)
- Hands-on experience with PyTorch (or similar) and model deployment
- Familiarity with GPU performance tuning (CUDA, ROCm, Triton, or kernel-level optimizations)
- Experience scaling inference for real users (not just research benchmarks)
- Comfortable working in fast-moving startup environments with ownership and ambiguity
Nice to Have
- Experience with LLM or long-context model inference
- Knowledge of inference frameworks (TensorRT, ONNX Runtime, vLLM, Triton)
- Experience optimizing across different hardware vendors
- Open-source contributions in ML systems or inference tooling
- Background in distributed systems or low-latency services
Why Join Us
- Real ownership over performance-critical systems
- Direct impact on product reliability and unit economics
- Close collaboration with research, infra, and product
- Competitive compensation + meaningful equity at Series A
- A team that cares about engineering quality, not hype
Originally posted on Himalayas