Product Manager - AI Inference & Model Serving

Mirantis

Where
Remote (Austin, TX, United States)

Posted 5 months ago· Checked live at Mirantis 1 hour ago

Stale: open 176 daysUp for more than two weeks. It may already be filled.

See how you score for this job

One sentence and your resume. We score it against what Mirantis asks for, line by line.

See my score
Is this job still open?
Yes. We checked it live on Mirantis's own hiring page 1 hour ago.
What is the job?
Job Description Mirantis is looking for a commercially driven, deeply technical Product Manager to own AI inference and model serving for k0rdent AI, our control plane for GPU infrastructure and distributed AI workloads. This role sits at the intersection of AI inference, cloud-native infrastructure, distributed systems, and performance engineering.
What does it pay?
The posting doesn't say.
Our verdict
Stale. Up for more than two weeks. It may already be filled.

What the job is

Job Description

Mirantis is looking for a commercially driven, deeply technical Product Manager to own AI inference and model serving for k0rdent AI, our control plane for GPU infrastructure and distributed AI workloads. This role sits at the intersection of AI inference, cloud-native infrastructure, distributed systems, and performance engineering.

Responsibilities

  • Own product strategy, roadmap, and lifecycle for inference and model serving, including serverless inference, dedicated endpoints, autoscaling, routing, KV cache management, and the related observability
  • Lead deep technical discovery with NeoClouds, sovereign clouds, and enterprise platform teams, and translate findings into prioritized requirements and architecture direction
  • Partner with engineering on system design trade-offs across runtime integration, GPU scheduling, network, storage, and serving topology, including disaggregated serving and multi-model serving
  • Define positioning grounded in measurable outcomes: latency distributions, throughput per GPU, utilization, tail reliability, and cost per tokens
  • Drive go-to-market execution: pricing and packaging, reference architectures, sizing guides, PoC playbooks, and direct engagement with customers, analysts, and ecosystem partners

Qualifications

  • 7+ years in product management, technical product management, or a senior technical role owning AI/ML and inference product(s)
  • Strong understanding of production AI inference, including model serving, serverless execution, dedicated endpoints, autoscaling, routing, workload placement, observability, and reliability
  • Proven capability to reason about performance trade-offs across GPU, network, storage, orchestration, and runtime layers, and to translate low-level technical capability into business value such as TTFT, throughput per GPU, and TCO
  • Working knowledge of modern inference runtimes (vLLM, SGLang, TensorRT-LLM, Dynamo, Triton) and the optimization patterns that matter in production: continuous batching, KV cache management, cold starts, prefill versus decode, disaggregated serving, and multi-model serving
  • Credibility with engineering leaders and infrastructure operators, including comfort in production architecture reviews and technical commercial conversations with platform engineering buyers

Why you’ll love Mirantis

  • Build the token factory foundation for the AI cloud era, working directly with leading GPU cloud operators, NeoClouds, sovereign clouds, and AI-first enterprises
  • Collaborate with a world-class, distributed team committed to openness and technical excellence
  • Shape the product narrative and influence go-to-market success

Additional Information

Similar open jobs

Product manager jobs in Austin (73)Remote product manager jobs (732)Jobs in Austin (4551)