AI Inference Platform Engineer

DRW

Where
Chicago
Pay
$200K to $250K a year

Posted 50 days ago· Checked live at DRW 1 hour ago

Stale: open 50 daysUp for more than two weeks. It may already be filled.

See how you score for this job

One sentence and your resume. We score it against what DRW asks for, line by line.

See my score
Is this job still open?
Yes. We checked it live on DRW's own hiring page 1 hour ago.
What is the job?
DRW is a diversified trading firm with over 3 decades of experience bringing sophisticated technology and exceptional people together to operate in markets around the world. We value autonomy and the ability to quickly pivot to capture opportunities, so we operate using our own capital and trading at our own risk.
What does it pay?
$200K to $250K a year, as the posting states.
Our verdict
Stale. Up for more than two weeks. It may already be filled.

What the job is

DRW is a diversified trading firm with over 3 decades of experience bringing sophisticated technology and exceptional people together to operate in markets around the world. We value autonomy and the ability to quickly pivot to capture opportunities, so we operate using our own capital and trading at our own risk.

What You'll Do

  • Optimize LLM inference performance across modern NVIDIA GPU architectures and inference runtimes.
  • Build end-to-end performance profiling and observability to identify bottlenecks from individual GPU kernels through multi-node inference systems.
  • Design and optimize KV cache and distributed inference architectures, including caching, routing, memory tiering, and prefill/decode strategies.
  • Own day-0 model onboarding, determining the appropriate runtime, precision, sharding, memory, batching, cache policy, and serving configuration for new models.
  • Maintain validated performance profiles for important model and hardware combinations, including performance and quality regression testing.
  • Measure and monitor quality equivalence across serving configurations, including KV cache quantization, speculative decoding acceptance thresholds, precision choices, and model routing, so in-house serving can be trusted to match reference-model quality on production workloads.
  • Manage the production serving lifecycle of models, including versioning, compatibility, staging, canarying, promotion, rollback, and retirement.
  • Partner with SRE and platform teams to automate model deployment, distribution, production readiness, observability, and reliable operation across environments.
  • Optimize model placement, scaling, and resource allocation across the inference fleet to improve utilization and cost efficiency while meeting performance and reliability requirements.
  • Design and operate multi-tenant scheduling and isolation across shared GPU capacity, balancing latency SLOs, throughput, and priority across concurrent workloads.

Similar open jobs

Machine learning engineer jobs in Chicago (88)Jobs in Chicago (6815)