Software Engineering Director, Agentic Evaluations

G2

Where
Remote (US)

Posted 57 days ago· Checked live at G2 2 hours ago

Stale: open 57 daysUp for more than two weeks. It may already be filled.

See how you score for this job

One sentence and your resume. We score it against what G2 asks for, line by line.

See my score
Is this job still open?
Yes. We checked it live on G2's own hiring page 2 hours ago.
What is the job?
G2 is the world's largest and most trusted software marketplace. When you join G2, you’re joining the industry’s leading team that helps businesses reach their peak potential by powering decisions and strategies with trusted insights from real software users.
What does it pay?
The posting doesn't say.
Our verdict
Stale. Up for more than two weeks. It may already be filled.

What the job is

G2 is the world's largest and most trusted software marketplace. When you join G2, you’re joining the industry’s leading team that helps businesses reach their peak potential by powering decisions and strategies with trusted insights from real software users.

Summary of Responsibilities

As a Software Engineering Director, Agentic Evaluations, your responsibilities will center around delivery of useful and credible evaluations of agents offered by software vendors. You’ll enhance the system that performs validation of agent performance, and lead the core team behind it. You will leverage “agent-first” ways of working, applying agentic engineering techniques to deliver your work to production.

Minimum qualifications

We realize applying for jobs can feel daunting at times. Even if you don’t check all the boxes in the job description, we encourage you to apply anyway.

  • 10+ years of professional programming experience in backend or full-stack environments
  • 2+ years of experience directly managing engineers
  • Expert-level proficiency in backend development using languages like Python, Java/Kotlin, Typescript/Javascript or Go; strong proficiency in associated backend frameworks like FastAPI or Node.js http://node.js
  • Direct experience creating evaluations (evals) against a customer-facing agent, leveraging agent trajectory trace data and rubrics to measure an agent’s task completion rate, accuracy, correctness, or policy adherence
  • Direct experience using frontier models from OpenAI, Anthropic, or Google in LLM-as-a-judge applications
  • Regular use of coding agent harnesses like Claude Code, Codex, Opencode, or Pi as part of the daily development workflow
  • BS/BA degree in related field

Similar open jobs

Remote engineering manager jobs (249)