Lead Product Data Scientist, Agents

Klaviyo

Where
Boston, MA
Pay
$172K to $258K a year

Posted 20 days ago· Checked live at Klaviyo 1 hour ago

Stale: open 20 daysUp for more than two weeks. It may already be filled.

See how you score for this job

One sentence and your resume. We score it against what Klaviyo asks for, line by line.

See my score
Is this job still open?
Yes. We checked it live on Klaviyo's own hiring page 1 hour ago.
What is the job?
At Klaviyo, we value the unique backgrounds, experiences and perspectives each Klaviyo (we call ourselves Klaviyos) brings to our workplace each and every day. We believe everyone deserves a fair shot at success and appreciate the experiences each person brings beyond the traditional job requirements. If you’re a close but not exact match with the description, we hope you’ll still consider applying.
What does it pay?
$172K to $258K a year, as the posting states.
Our verdict
Stale. Up for more than two weeks. It may already be filled.

What the job is

At Klaviyo, we value the unique backgrounds, experiences and perspectives each Klaviyo (we call ourselves Klaviyos) brings to our workplace each and every day. We believe everyone deserves a fair shot at success and appreciate the experiences each person brings beyond the traditional job requirements. If you’re a close but not exact match with the description, we hope you’ll still consider applying.

Who You Are

  • You have 10+ years of experience in data science, product analytics, applied statistics, or a related quantitative discipline, including recent hands-on experience measuring AI, machine learning, or LLM-powered products in production.
  • You bring deep expertise in product measurement, experimentation, statistical modeling, and causal inference, with the ability to select and defend rigorous analytical approaches when traditional randomized experiments aren't feasible.
  • You have experience evaluating AI systems in production, including designing evaluation frameworks, analyzing agent interactions or conversations, leveraging human labeling or LLM-as-judge methodologies, and understanding how model, prompt, and tool changes influence product outcomes.
  • You're highly proficient in SQL and Python or R, with experience developing production-quality analytical solutions and familiarity with dbt and modern analytics engineering practices.
  • You take an AI-first approach to your work, actively leveraging coding agents, LLMs, and emerging tools to accelerate analytical workflows while understanding their limitations and ensuring the accuracy and reliability of their outputs.
  • You've demonstrated technical leadership by setting direction on complex analytical initiatives, mentoring other data scientists, reviewing methodologies, and influencing technical standards across teams.
  • You combine statistical rigor with strong product and business intuition, understanding how customer engagement, AI product performance, usage-based pricing, and unit economics translate into meaningful customer and business outcomes.
  • You're comfortable tackling ambiguous, emerging problems where established metrics or methodologies may not yet exist, and can develop practical frameworks that balance analytical rigor with business needs.
  • You have a track record of influencing senior stakeholders, aligning cross-functional teams around shared definitions and priorities, and translating complex findings into clear recommendations for technical and non-technical audiences.
  • You're a compelling communicator who can explain sophisticated analytical concepts, challenge assumptions constructively, and communicate uncertainty and trade-offs with confidence.
  • You thrive in fast-moving product environments, adapt quickly as priorities evolve, and know how to balance analytical depth with the urgency and iteration required to build and improve AI-powered products.
  • You have a degree in Statistics, Mathematics, Computer Science, Economics, Finance, Engineering, or another quantitative discipline, or an MBA with a quantitative emphasis. Equivalent hands-on experience in applied data science is equally valued.

Nice to Have

  • Experience measuring and optimizing AI agents, conversational AI, or LLM-powered products in a high-growth B2B SaaS or product-focused technology company.
  • Hands-on experience with agent observability and evaluation platforms such as Braintrust, LangSmith, or Arize, including working with agent traces, tool-call data, and evaluation pipelines.
  • Experience with usage-based or consumption-based pricing, credit models, AI infrastructure costs, or LLM unit economics.
  • Experience building governed metrics, semantic layers, metric trees, or standardized measurement frameworks used across multiple product teams.
  • Experience applying classification, clustering, or other advanced analytical methods to large volumes of unstructured text, customer conversations, or agent interactions.
  • Experience developing internal AI agents or AI-powered analytics tools that improve team productivity and decision-making.
  • Familiarity with marketing attribution, incremental measurement, or evaluating the impact of AI-driven customer engagement.
  • Background in marketing technology, ecommerce, CRM, customer-service software, or other data-driven SaaS products.

Why This Role Matters

AI agents are central to Klaviyo's product strategy and the future of how businesses engage with their customers. As Composer and Customer Agent become increasingly sophisticated, understanding whether these products are delivering meaningful value, how effectively they perform, and where to invest next will be critical to their long-term success.

In this role, you'll help establish how Klaviyo measures the effectiveness and business impact of AI agents, developing the frameworks, methodologies, and analytical foundations that inform some of our most important product, pricing, and investment decisions. Your work will directly influence how we improve agent quality, understand customer outcomes, and connect AI innovation to measurable business value.

This is an opportunity to combine deep technical ownership with meaningful strategic influence, working alongside a highly collaborative, AI-first data science team to solve emerging measurement challenges and shape how Klaviyo builds, evaluates, and scales its next generation of intelligent products.

Base Pay Range For US Locations

$172,000 - $258,000 USD

This role may require up to 10% travel for purposes such as new hire onboarding, client or partner work if applicable, team meetings, and industry events. Travel is coordinated in advance.

Get to Know Klaviyo

We’re Klaviyo (pronounced clay-vee-oh). We empower creators to own their destiny by making first-party data accessible and actionable like never before. We see limitless potential for the technology we’re developing to nurture personalized experiences in ecommerce and beyond. To reach our goals, we need our own crew of remarkable creators—ambitious and collaborative teammates who stay focused on our north star: delighting our customers. If you’re ready to do the best work of your career, where you’ll be welcomed as your whole self from day one and supported with generous benefits, we hope you’ll join us.

AI fluency at Klaviyo includes responsible use of AI (including privacy, security, bias awareness, and human-in-the-loop). We provide accommodations as needed.

By participating in Klaviyo’s interview process, you acknowledge that you have read, understood, and will adhere to our Guidelines for using AI in the Klaviyo interview Process. For more information about how we process your personal data, see our Job Applicant Privacy Notice.

Klaviyo is committed to a policy of equal opportunity and non-discrimination. We do not discriminate on the basis of race, ethnicity, citizenship, national origin, color, religion or religious creed, age, sex (including pregnancy), gender identity, sexual orientation, physical or mental disability, veteran or active military status, marital status, criminal record, genetics, retaliation, sexual harassment or any other characteristic protected by applicable law.

IMPORTANT NOTICE: Our company takes the security and privacy of job applicants very seriously. We will never ask for payment, bank details, or personal financial information as part of the application process. All our legitimate job postings can be found on our official career site. Please be cautious of job offers that come from non-company email addresses (@klaviyo.com), instant messaging platforms, or unsolicited calls.

By clicking "Submit Application" you consent to Klaviyo processing your Personal Data in accordance with our Job Applicant Privacy Notice. If you do not wish for Klaviyo to process your Personal Data, please do not submit an application. You can find our Job Applicant Privacy Notice here and here (FR).

Similar open jobs

Data scientist jobs in Boston (88)Jobs in Boston (7098)