Machine Learning DevOps - Cloud and Compute Cluster
通过引入世界上第一个适应和像人类一样思考的后变革器模型，Pathway 正在摇动人工智能的基础.Pathway 的突破性架构 (BDH) 比 Transformer 更好，并为企业提供了完全的视觉，了解该模式如何工作。将基本模型与市场上最快的数据处理引擎结合起来，Pathways 使企业能够超越增进优化，向真正背景化的经验驱动的智能发展。该公司受到北约，La Poste 和 F1赛车团队等组织信任。路径由联合创始人兼首席执行官苏珊娜·斯塔米罗夫斯卡领导，这是一位复杂性科学家，他创造了一个由人工智能先驱组成的团队，包括 CTOJan Chorowski 是第一个应用注意力到言论的人，并与谷歌大脑诺贝尔奖获得者 Goeff Hinton 合作。以及 CSOAdrian Kosowski,一名领先的计算机科学家和量子物理学家在20岁获得博士学位。该公司得到了 TQ Ventures 和 Lukasz Kaiser 等主要投资者和顾问的支持，其中包括 Transformer （GPT 中的 T）的共同作者和 OpenAI 推理模型的主要研究员。路径总部位于加利福尼亚州帕洛阿尔托。机会背后 我们目前正在寻找一个拥有云和计算集群管理，扩展基础设施和 Linux 管理的经验的机器学习开发系统。我们的开发，ML 培训和生产环境在云中，使用几个主要的云提供商。我们需要支持管理和自动化流程，并将基础设施扩大到不断增长的团队和生产需求。你将 · 优化 ML 培训与推断 （例如 GPU,分布式计算）的基础设施。· 自动化和维护 ML/LLM 管道 （数据摄入，培训，验证，部署）. · 管理模型版本，可再生性和追溯性。· 与太字节大的数据集工作。· 实施 ML 中心的 CI/CD 实践。·监测模型性能和数据漂移在生产中。· 与机器学习工程师，软件工程师和平台团队合作。该角色重点是运营机器学习模型，确保可扩展性，可靠性和自动化在整个 ML 生命周期内。我们正在寻找什么要求。·非常熟悉 Linux,脚本和集群配置脚本作为基本工作工具。· 职责管理人员对工作负载进行了专业技能。
查看英文原文
About Pathway
Pathway is shaking the foundations of artificial intelligence by introducing the world’s first post-transformer model that adapts and thinks just like humans.
Pathway’s breakthrough architecture (BDH) outperforms Transformer and provides the enterprise with full visibility into how the model works. Combining the foundational model with the fastest data processing engine on the market, Pathway enables enterprises to move beyond incremental optimization and toward truly contextualized, experience-driven intelligence. The company is trusted by organizations such as NATO, La Poste, and Formula 1 racing teams.
Pathway is led by co-founder & CEO Zuzanna Stamirowska, a complexity scientist who created a team consisting of AI pioneers, including CTO Jan Chorowski who was the first person to apply Attention to speech and worked with Nobel laureate Goeff Hinton at Google Brain, as well as CSO Adrian Kosowski, a leading computer scientist and quantum physicist who obtained his PhD at the age of 20.
The company is backed by leading investors and advisors, including TQ Ventures and Lukasz Kaiser, co-author of the Transformer (“the T” in ChatGPT) and a key researcher behind OpenAI’s reasoning models. Pathway is headquartered in Palo Alto, California.
The opportunity
We are currently searching for a Machine Learning DevOps with experience in cloud and compute cluster management, scaling infrastructures, and Linux administration.
Our development, ML training, and production environment is in the cloud, using several major cloud providers. We need support in managing and automating the processes, and scaling the infrastructure to growing team and production needs.
You Will
- Optimize infrastructure for ML training and inference (e.g., GPUs, distributed compute).
- Automate and maintain ML/LLM pipelines (data ingestion, training, validation, deployment).
- Manage model versioning, reproducibility, and traceability.
- Work with terabyte-large datasets.
- Implement ML-centric CI/CD practices.
- Monitor model performance and data drift in production.
- Collaborate with machine learning engineers, software engineers, and platform teams.
The role focuses on operationalizing machine learning models, ensuring scalability, reliability, and automation across the ML lifecycle.
Requirements
What We Are Looking For
- Very good familiarity with Linux, shell scripts, and cluster configuration scripts as the basic work tool.
- Proficiency in workload management, containerization and orchestration (Slurm, Docker, Kubernetes).
- Solid grasp of CI/CD tools and workflows (GitHub Actions, Jenkins, Gitlab CI, etc.).
- Cloud infrastructure knowledge (AWS, GCP, Azure) – especially in ML services (e.g., SageMaker Hyperpod, Vertex AI).
- Familiarity with monitoring/logging tools (Grafana, CloudWatch, Prometheus, Loki).
- Experience with infrastructure as code (Terraform, CloudFormation, cluster-toolkit).
- Experience with ML pipeline orchestration tools (e.g., MLflow, Kubeflow, Airflow, Metaflow).
- Programming skills in Python (with exposure to ML libraries like TensorFlow, PyTorch).
- Experience with cluster, systems, and networks administration.
- Willingness to learn.
This position holds a minimum requirement of a BSc in Computer Science or Information Technology.
We will generally favor candidates who have undertaken ambitious efforts in the past. For example, if you have made an accepted contribution to the Linux kernel, won an important bug bounty, supported an academic grid/cluster computing team in a scaling effort, or even won a sports championship, make sure to mention this in your application!
Benefits
Why You Should Apply
- Intellectually stimulating work environment. Be a pioneer: you get to work with realtime data processing & AI.
- Work in one of the hottest AI startups, with exciting career prospects. Team members are distributed across the world.
- Responsibilities and ability to make significant contribution to the company’ success
- Inclusive workplace culture
Further details
- Type of contract: Permanent employment contract
- Preferable joining date: Immediate.
- Compensation: based on profile and location.
- Location: Remote work. Possibility to work or meet with other team members in one of our offices: Palo Alto, CA; Paris, France or Wroclaw, Poland. Candidates based anywhere in the EU, United States, and Canada will be considered.
Originally posted on Himalayas