Tech Lead Manager, AI Evaluations

Glean

Where
Mountain View, CA; San Francisco, CA
Pay
$250K to $350K a year

Posted 37 days ago· Checked live at Glean 1 hour ago

Stale: open 37 daysUp for more than two weeks. It may already be filled.

See how you score for this job

One sentence and your resume. We score it against what Glean asks for, line by line.

See my score
Is this job still open?
Yes. We checked it live on Glean's own hiring page 1 hour ago.
What is the job?
Glean is the Work AI platform that helps everyone work smarter with AI. What began as the industry’s most advanced enterprise search has evolved into a full-scale Work AI ecosystem, powering intelligent Search, an AI Assistant, and scalable AI agents on one secure, open platform.
What does it pay?
$250K to $350K a year, as the posting states.
Our verdict
Stale. Up for more than two weeks. It may already be filled.

What the job is

Glean is the Work AI platform that helps everyone work smarter with AI. What began as the industry’s most advanced enterprise search has evolved into a full-scale Work AI ecosystem, powering intelligent Search, an AI Assistant, and scalable AI agents on one secure, open platform.

About you

  • 10+ years of experience in software engineering, ML infrastructure, data systems, or a related technical field. Experience must include building and operating production systems.
  • 5+ years of engineering or applied AI team management experience, with a track record of technical leadership, developing engineers and scientists, and delivering cross-functional projects.
  • Hands-on experience designing, building, or operating evaluation systems for LLM/GenAI products, including model-based evaluators or autograders; practical understanding of how to assess model, retrieval/RAG, assistant, or agent behavior.
  • Familiarity with evaluation methods and tradeoffs: curated and production-derived datasets, benchmark creation, deterministic metrics, rubric-based and pairwise judgments, human review, judge validation and calibration, human-model alignment, and statistical analysis of results.
  • Understanding of agent evaluation, including tool calling, multi-turn behavior, trajectory quality, task completion, and recovery from errors; experience with safety testing or red teaming is a plus.
  • Strong coding ability in Python or another production language, with sound engineering fundamentals in testing, APIs, data modeling, and reliable distributed systems.
  • Experience building scalable data and developer platforms, experiment pipelines, or CI/CD integrations; familiarity with cloud infrastructure, Kubernetes, data stores, and observability is beneficial.
  • Strong judgment about measurement quality, including dataset representativeness, judge calibration, reproducibility, uncertainty, and the risk of optimizing to a narrow metric.
  • Knowledge of public benchmarks across domains and capabilities is a plus.
  • Product-minded and collaborative: able to translate ambiguous quality goals into clear measures, communicate findings to technical and non-technical audiences, and influence roadmaps across teams.
  • MS/PhD in Computer Science, AI or a related field.

Compensation & Benefits

The standard base salary range for this position is $250,000 - $350,000 annually. Compensation offered will be determined by factors such as location, level, job-related knowledge, skills, and experience. Certain roles may be eligible for variable compensation, equity, and benefits.

We offer a comprehensive benefits package including competitive compensation, Medical, Vision, and Dental coverage, generous time-off policy, and the opportunity to contribute to your 401k plan to support your long-term goals. When you join, you'll receive a home office improvement stipend, as well as an annual education and wellness stipends to support your growth and wellbeing. We foster a vibrant company culture through regular events, and provide healthy lunches daily to keep you fueled and focused.

We’re committed to building and sustaining a diverse, inclusive workplace. We strive to attract and retain people with a wide range of backgrounds, experiences, and perspectives, and we do not discriminate on the basis of gender, ethnicity, sexual orientation, religion, civil or family status, age, disability, or race.

Similar open jobs

Jobs in San Francisco (11750)Jobs in San Jose (7796)