Senior Inference SW Engineer

hpe

Where
Spring, Texas, United States of America

Posted 44 days ago· Checked live at hpe 3 days ago

Stale: open 44 daysUp for more than two weeks. It may already be filled.

See how you score for this job

One sentence and your resume. We score it against what hpe asks for, line by line.

See my score
Is this job still open?
Yes. We checked it live on hpe's own hiring page 3 days ago.
What is the job?
Senior Inference SW Engineer This role has been designed as 'Hybrid' with a requirement that you will work on average 2 days per week from an HPE office.
What does it pay?
The posting doesn't say.
Our verdict
Stale. Up for more than two weeks. It may already be filled.

What the job is

Senior Inference SW Engineer

This role has been designed as 'Hybrid' with a requirement that you will work on average 2 days per week from an HPE office.

Hewlett Packard Enterprise is the global edge-to-cloud company advancing the way people live and work.

Responsibilities

  • Design, implement, and own major components of the LLM serving deployment, including engine integration, continuous batching, KV cache management and reuse, and quantized execution
  • Partner with inference engineering teams and contribute to improving time-to-first-token, inter-token latency, throughput per GPU, and P95/P99 tail latency
  • Build and operate distributed execution capabilities, including disaggregated prefill/decode, tensor and pipeline parallelism, and KV cache offload across GPU memory, host memory, and RDMA-attached storage
  • Evaluate emerging runtimes, quantization schemes, speculative decoding, and mixture-of-experts serving, and make well-supported recommendations on adoption
  • Contribute to the orchestration layer supporting the runtime, including model admission, GPU scheduling and partitioning, cache-aware request routing, and autoscaling
  • Triage and resolve customer issues end-to-end, identifying root causes and improving systems and processes to prevent recurrence
  • Provide insightful code and design reviews, mentor team members, and lead by example on engineering practices within the team

Knowledge and Skills Required

  • Familiar with LLM inference engines such as vLLM, SGLang, TensorRT-LLM, TGI, or NVIDIA NIM, including modification of engine internals
  • Strong understanding of inference internals, including continuous batching, paged attention, KV cache reuse and prefix caching, chunked prefill, quantization, and speculative decoding
  • Working knowledge of tensor and pipeline parallelism, NCCL collective operations, and the GPU memory hierarchy and interconnect characteristics that govern them
  • Advanced proficiency in Kubernetes platform architectures, including operators, custom resources, controllers, and scheduling
  • Strong programming proficiency in Go and Python, with the ability to read, debug, and profile C++/CUDA using tools such as Nsight
  • Familiar with debugging/profiling multi-tier application workloads such as RAG, Agents
  • Excellent analytical, debugging, and problem-solving abilities

Preferred

  • Upstream contribution to vLLM, SGLang, TensorRT-LLM, llm-d, LMCache, or KServe
  • Disaggregated prefill/decode serving, or KV cache offload and reuse at scale
  • RDMA, GPUDirect Storage, InfiniBand, or RoCE
  • MIG, fractional GPU allocation, and multi-tenant GPU isolation
  • On-premises, air-gapped, or regulated enterprise software delivery

Experience and Education

  • Minimum of 8 years of experience in Software Engineering, including 1-2+ years working directly on LLM inference runtimes or production model serving
  • Degree in Computer Science or related field

Accessibility

HPE is committed to creating an inclusive and accessible workplace and encourages applications from all qualified individuals, including those with disabilities. If you believe you require accommodation during any stage of the application or interview process, please submit your request by completing our secure form linked here.

Note: This option is reserved for applicants needing assistance/reasonable accommodation related to a disability.