Principal Engineer, Distributed Systems

CoreWeave

Where
New York, NY / Sunnyvale, CA
Pay
$227K to $303K a year
Type
Full time

Posted 2 months ago· Checked live at CoreWeave 1 hour ago

Stale: open 65 daysUp for more than two weeks. It may already be filled.

See how you score for this job

One sentence and your resume. We score it against what CoreWeave asks for, line by line.

See my score
Is this job still open?
Yes. We checked it live on CoreWeave's own hiring page 1 hour ago.
What is the job?
CoreWeave is The Essential Cloud for AI™. Built for pioneers by pioneers, CoreWeave delivers a platform of technology, tools, and teams that enables innovators to build and scale AI with confidence. Trusted by leading AI labs, startups, and global enterprises, CoreWeave combines superior infrastructure performance with deep technical expertise to accelerate breakthroughs and turn compute into capability.
What does it pay?
$227K to $303K a year, as the posting states.
Our verdict
Stale. Up for more than two weeks. It may already be filled.

What the job is

CoreWeave is The Essential Cloud for AI™. Built for pioneers by pioneers, CoreWeave delivers a platform of technology, tools, and teams that enables innovators to build and scale AI with confidence. Trusted by leading AI labs, startups, and global enterprises, CoreWeave combines superior infrastructure performance with deep technical expertise to accelerate breakthroughs and turn compute into capability.

What you will do

  • Establish architectural approaches for distributed systems - multi-region operation, including service placement, failover, replication, traffic management, disaster recovery, and regional independence.
  • Drive decisions around consistency models, caching, invalidation, propagation, revocation, idempotency, concurrency, and ordering where correctness and security are critical.
  • Design for data residency, tenant isolation, trust boundaries, blast-radius reduction, and controlled handling of sensitive security data.
  • Define fault-tolerance strategies for dependencies, networks, regions, storage systems, and control-plane components, including graceful degradation and safe recovery.
  • Design systems that meet four-nines availability goals while maintaining predictable low latency and high throughput under normal operation, traffic spikes, and partial failures.
  • Improve the reliability and operability of critical services through SLOs, error budgets, metrics, logs, traces, audit events, alerting, incident response, and post-incident learning.
  • Guide teams through architecture reviews, design reviews, implementation tradeoffs, capacity planning, load testing, performance analysis, and production readiness assessments.
  • Mentor senior and staff engineers, develop technical talent, and raise the quality of engineering practice across the organization.
  • Communicate architecture, tradeoffs, risks, and recommendations clearly to technical and executive audiences.
  • Lead the design of high-scale authorization systems that support policy authoring, policy evaluation, access-control enforcement, auditability, and integration across many services and tenants.
  • Provide technical direction for a Security Token Service, including token issuance, validation, lifecycle management, trust relationships, key rotation, revocation, and secure service-to-service access.
  • Guide the design and implementation of an API Authentication Gateway that provides consistent, secure, observable authentication and authorization for CoreWeave APIs.
  • Set technical standards for API design, service contracts, threat modeling, cryptographic key management, secrets handling, observability, and operational readiness.
  • Partner with service teams to make authorization and authentication capabilities easy to adopt through clear interfaces, SDKs, reference implementations, documentation, and reliable integration patterns.

What you bring

  • 12+ years of experience designing and building production software, including substantial experience with distributed systems and platform infrastructure.
  • A track record of serving as a principal, staff-plus, distinguished, or equivalent technical leader across multiple teams or a broad technical domain.
  • Deep expertise in distributed-systems design, including scalability, availability, latency, consistency, concurrency, partition tolerance, caching, replication, and failure recovery.
  • Experience designing and operating security-sensitive or reliability-critical services in production.
  • Strong software engineering experience with one or more systems languages or backend ecosystems, such as Go, Java, Rust, C++, Python, or equivalent.
  • Demonstrated ability to move from ambiguous requirements to clear architecture, sequenced execution plans, and durable engineering outcomes.
  • Experience making architecture decisions that balance security, correctness, performance, operational simplicity, and time to value.
  • Strong written and verbal communication skills, including the ability to influence without direct authority and build alignment across organizational boundaries.
  • A track record of improving engineering standards, mentoring senior engineers, and helping teams deliver complex systems safely.

Preferred qualifications

  • Experience designing and implementing authorization systems at high scale, including policy engines, permission models, policy distribution, decision services, enforcement points, or authorization observability.
  • Experience designing or operating a Security Token Service, identity platform, credential service, or comparable security-critical control plane.
  • Experience designing or operating an API Authentication Gateway, service-mesh authorization layer, or centralized authentication and authorization platform.
  • Experience operating systems with four-nines availability requirements, high request volume, strict latency objectives, and demanding reliability expectations.
  • Experience with multi-region architecture, regional failover, active-active or active-passive operation, replication, disaster recovery, and regional isolation.
  • Experience reasoning about strong, eventual, and bounded-staleness consistency models, especially for security policy, credentials, tokens, permissions, and revocation.
  • Experience designing for data residency, regional data controls, tenant isolation, trust-domain separation, and compliance-sensitive deployment models.
  • Experience building systems that remain secure and useful during dependency failures, network partitions, regional outages, stale data, degraded capacity, or partial compromise.
  • Experience with authentication and identity standards such as OAuth 2.0, OpenID Connect, SAML, SCIM, JWT, mTLS, SPIFFE/SPIRE, or related protocols.
  • Experience with cryptography, key management, HSMs, secrets management, certificate authorities, signing keys, token validation, or secure credential lifecycle management.
  • Experience with Kubernetes, cloud infrastructure, service networking, distributed storage, messaging systems, and multi-cluster operations.
  • Experience building security and platform services with strong observability, including metrics, logs, traces, audit records, security events, and actionable alerting.
  • Experience working with enterprise customers, regulated workloads, or hybrid and multi-cloud environments.

How you will be successful

Why join Security Products

Security Products is building the systems that make CoreWeave trustworthy at scale. You will work on foundational infrastructure that protects access to cloud resources, enables secure product experiences, and supports customers running critical AI workloads.

This is an opportunity to shape the architecture of a modern cloud security platform while solving problems at the intersection of distributed systems, security engineering, reliability, performance, and global-scale infrastructure.

Location requirement

This position is based in Sunnyvale, California or New York, New York and requires working on-site. Remote work is not available for this role.

CoreWeave is an equal opportunity employer. We evaluate qualified applicants without regard to legally protected characteristics.

The base salary range for this role is $227,000 to $303,000. The starting salary will be determined based on job-related knowledge, skills, experience, and market location. We strive for both market alignment and internal equity when determining compensation. In addition to base salary, our total rewards package includes a discretionary bonus, equity awards, and a comprehensive benefits program (all based on eligibility).

What We Offer

In addition to a competitive salary, we offer a variety of benefits to support your needs. The benefits below reflect our US-based offerings for full-time employees; for roles in other locations, benefits vary and are shared during the hiring process. These include:

  • Medical, dental, and vision insurance - 100% paid for by CoreWeave
  • Company-paid Life Insurance
  • Voluntary supplemental life insurance
  • Short and long-term disability insurance
  • Flexible Spending Account
  • Health Savings Account
  • Tuition Reimbursement
  • Ability to Participate in Employee Stock Purchase Program (ESPP)
  • Mental Wellness Benefits through Spring Health
  • Family-Forming support provided by Carrot
  • Paid Parental Leave
  • Flexible, full-service childcare support with Kinside
  • 401(k) with a generous employer match
  • Flexible PTO
  • Catered lunch each day in our office and data center locations
  • A casual work environment
  • A work culture focused on innovative disruption

California Applicants

Similar open jobs

Jobs in New York (19157)Jobs in San Jose (7796)