Senior Systems Software Engineer - Accelerated Spark Applications
NVIDIA
- Where
- US, IL, Champaign
Posted 20 days ago· Checked live at NVIDIA 3 days ago
Stale: open 20 daysUp for more than two weeks. It may already be filled.
See how you score for this job
One sentence and your resume. We score it against what NVIDIA asks for, line by line.
See my score- Is this job still open?
- Yes. We checked it live on NVIDIA's own hiring page 3 days ago.
- What is the job?
- NVIDIA has been transforming computer graphics, PC gaming, and accelerated computing for more than 25 years. It’s a unique legacy of innovation that’s fueled by great technology—and amazing people. Today, we’re tapping into the unlimited potential of AI to define the next era of computing. An era in which our GPU acts as the brains of computers, robots, and self-driving cars that can understand the world.
- What does it pay?
- The posting doesn't say.
- Our verdict
- Stale. Up for more than two weeks. It may already be filled.
What the job is
NVIDIA has been transforming computer graphics, PC gaming, and accelerated computing for more than 25 years. It’s a unique legacy of innovation that’s fueled by great technology—and amazing people. Today, we’re tapping into the unlimited potential of AI to define the next era of computing. An era in which our GPU acts as the brains of computers, robots, and self-driving cars that can understand the world.
NVIDIA has been transforming computer graphics, PC gaming, and accelerated computing for more than 25 years. It’s a unique legacy of innovation that’s fueled by great technology—and amazing people. Today, we’re tapping into the unlimited potential of AI to define the next era of computing. An era in which our GPU acts as the brains of computers, robots, and self-driving cars that can understand the world. Doing what’s never been done before takes vision, innovation, and the world’s best talent. As an NVIDIAN, you’ll be immersed in a diverse, supportive environment where everyone is inspired to do their best work. Come join the team and see how you can make a lasting impact on the world.
NVIDIA is seeking a Senior Systems Software Engineer for the Apache Spark Acceleration group. GPU accelerated data processing has moved from proof of concept to production deployments. Enterprises now recognize the need for accelerated computing to handle large data processing needs. Multi-node GPU deployments reduce cloud computing costs and lower the latency of batch ETL workloads. At NVIDIA, we accelerate Apache Spark with an open source plugin that speeds up Spark applications on GPUs without code changes, and our cuDF Spark library is integrated with AWS EMR, Databricks, Google Dataproc, Oracle Cloud Data Flow, Bytedance Volcengine, Tencent Cloud and Cloudera. This role is on the team that runs that acceleration as a handled Spark service inside NVIDIA. The service accomplishes production data pipelines for NVIDIA’s internal data platform at thousands of jobs an hour on Kubernetes and AWS, under a customer SLA. It packages the cuDF Spark plugin and a CPU/GPU workload router into an API-based service built on Spark Connect, Kyuubi, Helm and Flux. You will work across all three layers: the service, the GPU Spark runtime, and the router.
What you’ll be doing
- Design, build and operate hosted Spark service control plane on Kubernetes: the Java REST API, the Spark Connect runner, Kyuubi, and deployment using Helm charts and Flux
- Own code review for one or more services end to end and keep them releasable on the release train
- Extend the cuDF Spark runtime and the CPU/GPU workload router in Scala and Java, including interoperability with Delta Lake, Apache Iceberg and Unity Catalog
- Investigate production failures on clusters running more than a thousand concurrent jobs together with the customer team and upstream library teams, and drive fixes to closure
- Instrument metrics, dashboards and alerts; take part in the on-call rotation and keep the runbooks current
- Provide recommendations and feedback to teams regarding decisions surrounding topics such as infrastructure, continuous integration and testing strategy
What we need to see
- BS, MS, or PhD in Computer Science, Computer Engineering, or a closely related field (or equivalent experience)
- 9+ years of experience in software development
- Strong distributed-systems fundamentals and proficiency in Java or Scala, plus C++ or Python
- Hands-on experience operating services on Kubernetes, on-premise or cloud
- Experience owning or reviewing a long-lived server: REST front ends, proxies or request routers, JDBC or Thrift services
- Experience supporting enterprise or internal production customers, including on-call
- Experience using AI tools in software development
NVIDIA has been transforming computer graphics, PC gaming, and accelerated computing for more than 25 years. It’s a unique legacy of innovation that’s fueled by great technology—and amazing people. Today, we’re tapping into the unlimited potential of AI to define the next era of computing. An era in which our GPU acts as the brains of computers, robots, and self-driving cars that can understand the world. Doing what’s never been done before takes vision, innovation, and the world’s best talent. As an NVIDIAN, you’ll be immersed in a diverse, supportive environment where everyone is inspired to do their best work. Come join the team and see how you can make a lasting impact on the world.
NVIDIA is seeking a Senior Systems Software Engineer for the Apache Spark Acceleration group. GPU accelerated data processing has moved from proof of concept to production deployments. Enterprises now recognize the need for accelerated computing to handle large data processing needs. Multi-node GPU deployments reduce cloud computing costs and lower the latency of batch ETL workloads. At NVIDIA, we accelerate Apache Spark with an open source plugin that speeds up Spark applications on GPUs without code changes, and our cuDF Spark library is integrated with AWS EMR, Databricks, Google Dataproc, Oracle Cloud Data Flow, Bytedance Volcengine, Tencent Cloud and Cloudera. This role is on the team that runs that acceleration as a handled Spark service inside NVIDIA. The service accomplishes production data pipelines for NVIDIA’s internal data platform at thousands of jobs an hour on Kubernetes and AWS, under a customer SLA. It packages the cuDF Spark plugin and a CPU/GPU workload router into an API-based service built on Spark Connect, Kyuubi, Helm and Flux. You will work across all three layers: the service, the GPU Spark runtime, and the router.
What you’ll be doing
- Design, build and operate hosted Spark service control plane on Kubernetes: the Java REST API, the Spark Connect runner, Kyuubi, and deployment using Helm charts and Flux
- Own code review for one or more services end to end and keep them releasable on the release train
- Extend the cuDF Spark runtime and the CPU/GPU workload router in Scala and Java, including interoperability with Delta Lake, Apache Iceberg and Unity Catalog
- Investigate production failures on clusters running more than a thousand concurrent jobs together with the customer team and upstream library teams, and drive fixes to closure
- Instrument metrics, dashboards and alerts; take part in the on-call rotation and keep the runbooks current
- Provide recommendations and feedback to teams regarding decisions surrounding topics such as infrastructure, continuous integration and testing strategy
What we need to see
- BS, MS, or PhD in Computer Science, Computer Engineering, or a closely related field (or equivalent experience)
- 9+ years of experience in software development
- Strong distributed-systems fundamentals and proficiency in Java or Scala, plus C++ or Python
- Hands-on experience operating services on Kubernetes, on-premise or cloud
- Experience owning or reviewing a long-lived server: REST front ends, proxies or request routers, JDBC or Thrift services
- Experience supporting enterprise or internal production customers, including on-call
- Experience using AI tools in software development
Ways to stand out from the crowd
- Apache Spark internals, Spark Connect, or Kyuubi
- Delta Lake, Apache Iceberg or Unity Catalog integration work
- GPU data processing with cuDF, or CUDA experience
- Significant contributions to data platform projects committed to collaboration and shared development (Apache Spark, Velox, Presto, Apache Arrow, Apache DataFusion)
Widely Is considered to be one of the technology world’s most desirable employers, NVIDIA offers highly competitive salaries and a comprehensive benefits package. As you plan your future, see what we can offer to you and your family www.nvidiabenefits.com/
Your base salary will be determined based on your location, experience, and other objective job-related factors. The expected base salary range for this position is 184,000 USD – 287,500 USD.
You will also be eligible for equity and benefits.
Applications for this job will be accepted at least until November 3, 2026.
This posting is for an existing vacancy.
NVIDIA uses AI tools in its recruiting processes.
NVIDIA is committed to fostering an inclusive work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.
Similar open jobs
- Staff Engineer, CPU Load Store VerificationTenstorrent · Austin, Texas, United States · $100K to $500K a year
- System Software Lab Integration EngineerIntuitive · Sunnyvale, CA, United States
- Staff Electrical EngineerCoreWeave · New York, NY / Sunnyvale, CA / Bellevue, WA · $207K to $275K a year
- Senior Software Engineer, Cloud PaymentsMongoDB · Atlanta · $126K to $248K a year
- Staff Engineer, Design VerificationSamsung Semiconductor · San Jose, California, United States · $163K to $253K a year
- Staff Software Engineer - Threat and Abuse DefenseDatabricks · Mountain View, California · $181K to $203K a year
Software engineer jobs in San Jose (1462)Jobs in San Jose (7796)