Machine Learning Systems Administrator / DevOps Engineer: Training Clusters & Inference Cloud

Zyphra

Where
San Francisco

Posted 34 days ago· Checked live at Zyphra 1 hour ago

Stale: open 34 daysUp for more than two weeks. It may already be filled.

See how you score for this job

One sentence and your resume. We score it against what Zyphra asks for, line by line.

See my score
Is this job still open?
Yes. We checked it live on Zyphra's own hiring page 1 hour ago.
What is the job?
As a Machine Learning Systems Administrator / DevOps Engineer, you will maintain and build the core infrastructure behind both our model training and our inference cloud. You’ll split your time between our large-scale training clusters and the deployments and services that run our models in production on heterogeneous compute, which spans multiple accelerator types and vendors.
What does it pay?
The posting doesn't say.
Our verdict
Stale. Up for more than two weeks. It may already be filled.

What the job is

As a Machine Learning Systems Administrator / DevOps Engineer, you will maintain and build the core infrastructure behind both our model training and our inference cloud. You’ll split your time between our large-scale training clusters and the deployments and services that run our models in production on heterogeneous compute, which spans multiple accelerator types and vendors.

What We’re Looking For / Requirements

  • Experience with Linux system administration, user and access management, and automation
  • Demonstrated expertise in scripting languages for system tooling and automation (bash, Python, etc.)
  • Experience with containers (Docker) and scheduling/orchestration systems (Slurm, Kubernetes)
  • Experience running production services, including monitoring and alerting (e.g., Prometheus, Grafana), deployment pipelines, and incident response
  • Experience building tooling for cluster validation and reliability (accelerator, networking, and storage health checks)
  • Experience setting up and managing developer tools and third-party services (e.g., cloud storage providers, Docker Hub, Slack, Gmail, Telegraf, experiment trackers)
  • Excellent debugging and troubleshooting skills across compute, storage, and networking
  • Strong communication skills and ability to collaborate across technical and non-technical teams

Qualifications / Additional Skills

  • Experience with infrastructure as code (e.g., Ansible, Terraform)
  • Prior work supporting ML/AI infrastructure, including GPU management and workload optimization
  • Experience deploying and operating LLM serving stacks (e.g., vLLM, SGLang, Ray)
  • Experience with multiple accelerator platforms (e.g., NVIDIA CUDA, AMD ROCm, TPUs)
  • Experience with high-performance networking (InfiniBand, RoCE) and distributed storage
  • Experience working with cloud platforms such as AWS, Azure, or GCP
  • Familiarity with Apptainer and with integrating containers into Slurm and Kubernetes

Benefits and Perks

  • Comprehensive medical, dental, vision, and FSA plans
  • Competitive compensation and 401(k) plan
  • Relocation and immigration support on a case-by-case basis
  • In-office snacks and meals provided
  • Unlimited PTO and company holidays
  • In-person team in San Francisco with a collaborative, high-energy environment

Similar open jobs

Machine learning engineer jobs in San Francisco (586)Jobs in San Francisco (11750)