Engineering Manager, Deep Learning Inference

NVIDIA

Where
US, CA, Santa Clara

Posted 20 days ago· Checked live at NVIDIA 3 days ago

Stale: open 20 daysUp for more than two weeks. It may already be filled.

See how you score for this job

One sentence and your resume. We score it against what NVIDIA asks for, line by line.

See my score
Is this job still open?
Yes. We checked it live on NVIDIA's own hiring page 3 days ago.
What is the job?
NVIDIA is seeking an exceptional Manager, Deep Learning Inference Software, to lead a world-class engineering team advancing the state of AI model deployment. You will shape the software powering today’s most sophisticated AI systems — from large language models to multimodal generative AI — all accelerated on NVIDIA GPUs.
What does it pay?
The posting doesn't say.
Our verdict
Stale. Up for more than two weeks. It may already be filled.

What the job is

NVIDIA is seeking an exceptional Manager, Deep Learning Inference Software, to lead a world-class engineering team advancing the state of AI model deployment. You will shape the software powering today’s most sophisticated AI systems — from large language models to multimodal generative AI — all accelerated on NVIDIA GPUs.

NVIDIA is seeking an exceptional Manager, Deep Learning Inference Software, to lead a world-class engineering team advancing the state of AI model deployment. You will shape the software powering today’s most sophisticated AI systems — from large language models to multimodal generative AI — all accelerated on NVIDIA GPUs. The Deep Learning Inference team develops and optimizes open-source frameworks that make AI deployment scalable, efficient, and accessible — including vLLM / SGLang, and FlashInfer. Our work enables developers worldwide to harness NVIDIA accelerators for real-time inference at every scale, from datacenter clusters to edge devices.

What you'll be doing

  • Lead, mentor, and scale a high-performing engineering team focused on deep learning inference and GPU-accelerated software.
  • Drive the strategy, roadmap, and execution of NVIDIA’s inference frameworks engineering, focusing on Client AI.
  • Partner with internal compiler, libraries, and research teams to deliver end-to-end optimized inference pipelines across NVIDIA accelerators.
  • Oversee performance tuning, profiling, and optimization of large-scale models for LLM, multimodal, and generative AI applications.
  • Guide engineers in adopting best practices for CUDA, Triton, CUTLASS, and multi-GPU communications (NIXL, NCCL, NVSHMEM).
  • Represent the team in roadmap and planning discussions, ensuring alignment with NVIDIA’s broader AI and software strategies.
  • Foster a culture of technical excellence, open collaboration, and continuous innovation.

What we need to see

  • MS, PhD, or equivalent experience in Computer Science, Electrical/Computer Engineering, or a related field.
  • 6+ overall years of software development experience, including 3+ years in technical leadership or engineering management.
  • Strong background in C/C++ software design and development; proficiency in Python is a plus.
  • Hands-on experience with GPU programming (CUDA, Triton, CUTLASS) and performance optimization.
  • Proven record of deploying or optimizing deep learning models in production environments.
  • Experience leading teams using Agile or collaborative software development practices.

Similar open jobs

Engineering manager jobs in San Jose (133)Jobs in San Jose (7796)