Staff / Principal Software Engineer, Inference Platform - USA

Inworld Ai

Where
Hybrid in Mountain View, California, USA
Pay
$280K to $350K a year
Type
Full time

Posted 8 months ago· Checked live at Inworld Ai 2 hours ago

Stale: open 246 daysUp for more than two weeks. It may already be filled.

See how you score for this job

One sentence and your resume. We score it against what Inworld Ai asks for, line by line.

See my score
Is this job still open?
Yes. We checked it live on Inworld Ai's own hiring page 2 hours ago.
What is the job?
Inworld is a research lab and inference provider focused on realtime AI for consumer-facing applications. We build first-party speech models, serve LLMs, and run the inference behind modular APIs designed for high-volume, realtime workloads. Hundreds of millions of users interact with Inworld powered apps every day and we serve over 10 trillion LLM tokens per month.
What does it pay?
$280K to $350K a year, as the posting states.
Our verdict
Stale. Up for more than two weeks. It may already be filled.

What the job is

Inworld is a research lab and inference provider focused on realtime AI for consumer-facing applications. We build first-party speech models, serve LLMs, and run the inference behind modular APIs designed for high-volume, realtime workloads.

Hundreds of millions of users interact with Inworld powered apps every day and we serve over 10 trillion LLM tokens per month.

What you'll do

  • Design and build the platform's core services: control plane, model and deployment lifecycle APIs, tenant isolation, metering and quotas.
  • Own how GPU capacity is allocated and scheduled across shared and reserved workloads, with clear latency, cost and reliability trade-offs.
  • Build the path that takes a customer's model from weights to production serving, and keeps it healthy as it changes.
  • Write production code, mostly in Go, every week. You ship services; you don't just configure them.
  • Define the APIs and abstractions that customers and internal teams build on. Make the right path the easy path.
  • Lead design reviews and write the technical direction for the platform. Raise the bar for system design across engineering.
  • Run what you build: set SLOs, take part in incidents, and turn postmortems into design changes.

What we're looking for

  • 8+ years building backend or distributed systems in Go, Rust, Java or C++.
  • You've designed and shipped a multi-service system end to end, and can explain its trade-offs and what broke in production.
  • Deep understanding of distributed systems: consistency, failure modes, queuing, load balancing and capacity modelling.
  • Experience building a platform or developer product with external customers: cloud compute, PaaS, serverless, or an API platform.
  • Clear technical writing. You can turn an ambiguous problem into a design doc that others build from.

Nice to have

  • Schedulers or resource managers for scarce hardware (GPU clusters, batch systems, cloud compute).
  • LLM inference serving (vLLM, SGLang, TensorRT-LLM) or ML training infrastructure.
  • Multi-tenant systems with usage-based billing or contractual SLAs.
  • Comfortable with Kubernetes, Terraform and GitOps (ArgoCD).

Similar open jobs

Software engineer jobs in San Jose (1462)Jobs in San Jose (7796)