Staff Engineer, Inference Optimizations

DigitalOcean

Where
Austin
Pay
$191K to $239K a year

Posted 3 months ago· Checked live at DigitalOcean 1 hour ago

Stale: open 118 daysUp for more than two weeks. It may already be filled.

See how you score for this job

One sentence and your resume. We score it against what DigitalOcean asks for, line by line.

See my score
Is this job still open?
Yes. We checked it live on DigitalOcean's own hiring page 1 hour ago.
What is the job?
Dive in and do the best work of your career at DigitalOcean. Journey alongside a strong community of top talent who are relentless in their drive to build the simplest scalable cloud. If you have a growth mindset, naturally like to think big and bold, and are energized by the fast-paced environment of a true industry disruptor, you’ll find your place here.
What does it pay?
$191K to $239K a year, as the posting states.
Our verdict
Stale. Up for more than two weeks. It may already be filled.

What the job is

Dive in and do the best work of your career at DigitalOcean. Journey alongside a strong community of top talent who are relentless in their drive to build the simplest scalable cloud. If you have a growth mindset, naturally like to think big and bold, and are energized by the fast-paced environment of a true industry disruptor, you’ll find your place here.

What You’ll Do

  • Performance Architecture: Lead the technical strategy for benchmarking and performance optimizations at the inference engine and GPU kernel layers, ensuring our infrastructure extracts maximum value from every TFLOP.
  • Deep-Dive Optimization: Engineer solutions for complex performance issues, including attention layer optimizations, memory and precision management, and advanced parallelization across multi-node GPU clusters.
  • Technological Innovation: Proactively implement cutting-edge optimization techniques to keep DigitalOcean at the forefront of the Gen AI landscape. Some examples of projects you may work on:
  • Improving batch size performance using AMD's AITER library for AMD MI355X - identify and tune AITER's CK (composable kernel) or ASK (assembly) to optimize FP8 / BF16
  • Identify kernel fusion opportunities for GLM-5 kernels for different layers of the Transformer block (FlashAttention, RMS Norm)
  • Tune expert gateway router kernels for MoE models like Qwen3-235B, DeepSeek V3, GLM-5 etc
  • Hardware & Ecosystem Mastery: Act as the subject matter expert on modern GPU families (NVIDIA/AMD) and their software stacks (CUDA, ROCm, TensorRT, OpenAI Triton), advising on hardware procurement and software integration.
  • Precision Optimization: Develop and deploy state-of-the-art quantization techniques (FP8, INT8, and experimental FP4 ) to double throughput without losing accuracy.
  • Technical Mentorship: Lead by example through high-quality code and design reviews, elevating the technical bar for the team without the administrative overhead of direct management.
  • Strategic Collaboration: Partner with Product Management and TPMs to translate "theoretical hardware limits" into "shippable product features," ensuring our platform is both powerful and developer-friendly.
  • Community Leadership: Maintain a strong presence in the GPU infrastructure and model performance optimization communities, contributing to and integrating the best of open-source AI.

Compensation Range

  • $191,200 - $239,000

*This is a remote role

JR: 2026-7625

Similar open jobs

Software engineer jobs in San Francisco (1709)Software engineer jobs in Seattle (980)Software engineer jobs in Austin (367)Software engineer jobs in Boston (368)Software engineer jobs in Denver (281)Jobs in San Francisco (11750)Jobs in Seattle (5777)Jobs in Austin (4551)Jobs in Boston (7098)Jobs in Denver (3072)