Software Engineer, Inference

Sierra

Where
San Francisco, CA
Type
Full time

Posted 44 days ago· Checked live at Sierra 1 hour ago

Stale: open 44 daysUp for more than two weeks. It may already be filled.

See how you score for this job

One sentence and your resume. We score it against what Sierra asks for, line by line.

See my score
Is this job still open?
Yes. We checked it live on Sierra's own hiring page 1 hour ago.
What is the job?
Sierra is the leading platform for customer-facing AI agents, working with many of the world's biggest brands — including The GAP, Rocket Mortgage, SoFi, Sutter Health, and SoftBank — to transform how they serve customers and grow their businesses. We are primarily an in-person company based in San Francisco, with growing offices across North America, Europe, and Asia.
What does it pay?
The posting doesn't say.
Our verdict
Stale. Up for more than two weeks. It may already be filled.

What the job is

Sierra is the leading platform for customer-facing AI agents, working with many of the world's biggest brands — including The GAP, Rocket Mortgage, SoFi, Sutter Health, and SoftBank — to transform how they serve customers and grow their businesses. We are primarily an in-person company based in San Francisco, with growing offices across North America, Europe, and Asia.

What you'll do

  • Partner with frontier labs and providers. At our scale, we rely on frontier labs, and inference providers to supply capacity, training and inference infrastructure.
  • Shape Sierra’s inference architecture. Design how inference traffic flows across models, infrastructure, and providers, including new serving and proxy layers as Sierra scales.
  • Build for low latency and high reliability. Develop systems for routing, failover, capacity management, and quota that keep inference performant and available across large-scale production workloads.
  • Build and operate self-hosted inference. Run models on GPU infrastructure, from building containers and operating inference engines to managing the underlying compute capacity.
  • Optimize inference performance. Work with the Applied Research team on techniques such as speculative decoding and serving-engine optimizations that improve latency, throughput, and cost.
  • Build across a hybrid inference stack. Work with both Sierra-managed infrastructure and leading inference platforms, making architectural decisions about where and how workloads should run.
  • Push the serving stack forward. Work closely with inference providers to tune engines and infrastructure for Sierra’s workloads.
  • Support the broader model lifecycle. Contribute to infrastructure that enables post-training while partnering closely with our Models and Agent Runtime teams.

Even better

  • Experience with ML infrastructure, MLOps, or production inference systems.
  • Experience serving LLMs or other large models at scale.
  • Experience operating self-hosted inference and GPU infrastructure.
  • Familiarity with inference frameworks such as vLLM or SGLang.
  • Experience with post-training infrastructure or inference-performance optimization.

What we offer

We want our benefits to reflect our values and offer the following to full-time employees:

  • Flexible (unlimited) paid time off
  • Medical, dental, and vision benefits for you and your family
  • Life insurance and disability benefits
  • Retirement plan dependent on country of employment
  • Parental leave
  • Fertility and family building benefits
  • Lunch, as well as delicious snacks and coffee to keep you energized when working onsite
  • Discretionary benefit stipend giving people the ability to spend where it matters most
  • Free alphorn lessons

These benefits are further detailed in Sierra's policies, may vary by region and employment type, and are subject to change at any time, consistent with the terms of any applicable compensation or benefits plans. Eligible full-time employees can participate in Sierra's equity plans subject to the terms of the applicable plans and policies.

Similar open jobs

Software engineer jobs in San Francisco (1709)Jobs in San Francisco (11750)