Senior Software Engineer, Spark Platform

DoorDash USA

Where
San Francisco, CA; Sunnyvale, CA; Seattle, WA; New York City, NY
Pay
$160K to $235K a year

Posted 37 days ago· Checked live at DoorDash USA 3 hours ago

Stale: open 37 daysUp for more than two weeks. It may already be filled.

See how you score for this job

One sentence and your resume. We score it against what DoorDash USA asks for, line by line.

See my score
Is this job still open?
Yes. We checked it live on DoorDash USA's own hiring page 3 hours ago.
What is the job?
The Spark Platform team owns and operates DoorDash's Apache Spark ecosystem — the execution runtime, remote shuffle service, cluster scheduler, and reliability tooling that powers the company's data, analytics, and ML workloads. We run Spark across the company at significant scale and continue to expand the workloads, capabilities, and consumer base we serve.
What does it pay?
$160K to $235K a year, as the posting states.
Our verdict
Stale. Up for more than two weeks. It may already be filled.

What the job is

The Spark Platform team owns and operates DoorDash's Apache Spark ecosystem — the execution runtime, remote shuffle service, cluster scheduler, and reliability tooling that powers the company's data, analytics, and ML workloads. We run Spark across the company at significant scale and continue to expand the workloads, capabilities, and consumer base we serve.

We're excited about you because…

  • B.S., M.S., or PhD in Computer Science or equivalent.
  • 6+ years of industry experience designing and operating distributed systems at scale.
  • Deep, hands-on experience with Apache Spark — internals, query execution, shuffle, the executor/driver model — at platform scale on Amazon EMR, Databricks, or an in-house deployment, with a focus on platform operations (runtime upgrades, cluster lifecycle, shuffle, observability, multi-tenant scheduling) rather than authoring individual Spark jobs.
  • Production experience with one or more of: remote/external shuffle systems (Celeborn, Magnet, Cosco, or similar), batch/big-data schedulers (YuniKorn, Volcano, Kueue, or the Spark-on-Kubernetes operator), or the observability and SRE patterns that make distributed compute platforms operable.
  • Strong fluency operating workloads on Kubernetes in production — operator patterns, executor pod lifecycle, network topology, and the multi-tenant failure modes that show up at scale.
  • Familiarity with data lake table formats such as Apache Iceberg or Delta Lake, and with the query and SQL engines that read them.
  • Track record of acting as a technical leader on a platform team — setting direction, mentoring, and partnering with management on roadmap and hiring.
  • Professional experience with Scala, Java, Python, or Go; strong SQL.
  • You are located or willing to relocate to the Bay Area, Seattle, or NYC.

Applications for this position are accepted on an ongoing basis

Compensation

The successful candidate’s starting pay will fall within the pay range listed below and is determined based on job-related factors including, but not limited to, skills, experience, qualifications, work location, and market conditions. Base salary is localized according to an employee’s work location. Ranges are market-dependent and may be modified in the future.

In addition to base salary, the compensation for this role includes opportunities for equity grants. Talk to your recruiter for more information.

DoorDash cares about you and your overall well-being. That’s why we offer a comprehensive benefits package to all regular employees, which includes a 401(k) plan with employer matching, 16 weeks of paid parental leave, wellness benefits, commuter benefits match, paid time off and paid sick leave in compliance with applicable laws (e.g. Colorado Healthy Families and Workplaces Act). DoorDash also offers medical, dental, and vision benefits, 11 paid holidays, disability and basic life insurance, family-forming assistance, and a mental health program, among others.

To learn more about our benefits, visit our careers page here.

Similar open jobs

Software engineer jobs in San Francisco (1709)Software engineer jobs in New York (1437)Software engineer jobs in Seattle (980)Software engineer jobs in San Jose (1462)Jobs in San Francisco (11750)Jobs in New York (19157)Jobs in Seattle (5777)Jobs in San Jose (7796)