Site Reliability Engineering Lead

Graphcore

Where
Austin, Texas, United States

Posted 50 days ago· Checked live at Graphcore 1 hour ago

Stale: open 50 daysUp for more than two weeks. It may already be filled.

See how you score for this job

One sentence and your resume. We score it against what Graphcore asks for, line by line.

See my score
Is this job still open?
Yes. We checked it live on Graphcore's own hiring page 1 hour ago.
What is the job?
How often do you get the chance to build a technology that transforms the future of humanity? Graphcore products have set the standard in made-for-AI compute hardware and software, gaining global attention and industry acclaim.
What does it pay?
The posting doesn't say.
Our verdict
Stale. Up for more than two weeks. It may already be filled.

What the job is

How often do you get the chance to build a technology that transforms the future of humanity?

Graphcore products have set the standard in made-for-AI compute hardware and software, gaining global attention and industry acclaim.

Responsibilities and Duties

  • Build the SRE Organization:
  • Build and develop the team from its initial formation through full 24x7x365 production operations, including defining roles, interviewing and hiring team members, establishing career expectations, and developing future technical leaders.
  • Mentor engineers and team leads, develop successors, and build an organization capable of operating effectively without depending on any single individual.
  • Work with leadership to forecast staffing requirements as the platform grows from initial deployment through full production scale.
  • Establish the Production Operating Model:
  • Define the operating model for an SRE organization, including staffing and coverage model, escalation paths, on-call responsibilities, incident management, handoffs, production access, change management, and operational readiness requirements.
  • Establish clear operational interfaces with Datacenter Operations, engineering teams, vendors, and other service owners.
  • Establish and continuously improve production readiness standards, runbooks, operational procedures, failure-mode documentation, escalation processes, and incident response practices with a strong emphasis on automation and engineering over manual operational work.
  • Develop training, cross-training, simulation, and production incident-response exercises to ensure the team can operate independently and confidently.
  • Engineer Reliability Into the Platform:
  • Embed with platform engineering teams during development to gain deep technical knowledge of the system and ensure reliability, serviceability, observability, and operational requirements are incorporated into the platform before production.
  • Lead the development of SLOs, operational health indicators, alerting standards, incident severity definitions, and reliability reporting appropriate for a large-scale production infrastructure service.
  • Build a culture in which recurring operational problems are engineered out through automation, improved observability, better platform design, and elimination of unnecessary toil.
  • Ensure the SRE organization can rapidly diagnose and mitigate issues across compute, networking, storage, and supporting infrastructure.
  • Develop strong technical depth within the team while maintaining access to specialist expertise in critical areas such as high-performance networking, storage, observability, and platform scheduling.
  • Lead Production Operations:
  • Lead or participate in major production incidents as necessary, particularly during platform development, launch, and early production.
  • Establish a blameless post-incident review process focused on identifying systemic improvements and ensuring corrective actions are completed.
  • Serve as the senior operational authority for the SRE organization and represent production reliability concerns in engineering and leadership discussions.

Required Skills and Experience

  • Significant experience leading or building an SRE, Production Engineering, Infrastructure Reliability, or comparable function supporting large-scale, highly available production infrastructure.
  • Experience taking a new or rapidly evolving platform through production readiness, launch, stabilization, and ongoing operation.
  • Experience building and operating sustainable 24x7x365 production support or on-call organizations.
  • Strong understanding of modern Site Reliability Engineering principles, including SLOs, incident management, observability, automation, toil reduction, capacity management, and production readiness.
  • Strong incident leadership experience, including managing high-severity, multi-team production incidents under significant time pressure.
  • A strong systems engineering background with a working understanding of Linux, networking, storage, distributed systems, automation, and production infrastructure.
  • Ability to operate effectively at both the leadership and hands-on technical levels, moving comfortably between organizational design, architecture discussions, troubleshooting, and incident response.
  • Demonstrated ability to lead, hire, mentor, develop, and retain a growing team of strong technical engineers at various levels of capability
  • Strong judgment regarding when to solve an immediate operational problem and when to invest in eliminating the underlying failure mode.
  • Strong written and verbal communication skills, particularly during incidents and when communicating technical risk to senior leadership.
  • Ability to manage conflicting priorities under a pressured environment

Desired but Not Required

Candidates are not expected to have experience in all the areas below. Experience in several would be particularly valuable:

  • High-performance networking technologies such as InfiniBand, RDMA, RoCE, or large-scale Ethernet fabrics.
  • Large-scale parallel or distributed storage systems.
  • Workload schedulers and orchestration platforms such as Kubernetes, Slurm, or comparable systems.
  • Production observability and service-level monitoring for complex distributed infrastructure.
  • Custom or early-generation hardware, firmware, or environments where hardware and software are developed concurrently.
  • Operational relationships and escalation processes involving datacenter operations and infrastructure/hardware/networking vendors.
  • Rapid capacity expansion, datacenter migration, or transitions between temporary and permanent production environments.

Similar open jobs

Site reliability engineer jobs in Austin (46)Jobs in Austin (4551)