Data Quality and Labeling Specialist

Collate

Where
San Francisco
Pay
$125 an hour

Posted 22 months ago· Checked live at Collate 1 hour ago

Stale: open 680 daysUp for more than two weeks. It may already be filled.

See how you score for this job

One sentence and your resume. We score it against what Collate asks for, line by line.

See my score
Is this job still open?
Yes. We checked it live on Collate's own hiring page 1 hour ago.
What is the job?
Collate is an AI document generation platform for life sciences. We automate paperwork with AI, helping our customers get life-saving innovations to patients years faster. Collate is an end-to-end solution, powering every step of drug, diagnostic, and medical device development—from concept to market. Our CEO Surbhi Sarna is a former General Partner at Y Combinator.
What does it pay?
$125 an hour, as the posting states.
Our verdict
Stale. Up for more than two weeks. It may already be filled.

What the job is

Collate is an AI document generation platform for life sciences. We automate paperwork with AI, helping our customers get life-saving innovations to patients years faster. Collate is an end-to-end solution, powering every step of drug, diagnostic, and medical device development—from concept to market.

Our CEO Surbhi Sarna is a former General Partner at Y Combinator.

What You’ll Do

  • Review and label regulatory, quality, and clinical documents and model-generated outputs for accuracy, completeness, and compliance.
  • Build and maintain gold-standard datasets used to train and evaluate our models.
  • Write and maintain labeling guidelines and evaluation rubrics as new document types and use cases are added.
  • Run calibration with other reviewers to keep labels consistent and resolve disagreements.
  • Audit labeled data before it reaches training and evaluation, and track label quality over time.
  • Turn recurring model errors into clear, actionable feedback for the research and engineering teams.

What We’re Looking For

  • Experience in a regulated life-sciences function such as regulatory affairs, quality, clinical, or CMC, at a pharma, biotech, diagnostics, or medical device company.
  • Familiarity with the documents and standards that govern that work (e.g., FDA, EMA, ICH, or ISO requirements).
  • Strong attention to detail: you notice an unsupported claim or inconsistent value and can explain why it matters.
  • Ability to apply structured rubrics consistently and to flag when a rubric itself needs changing.
  • Clear written communication.
  • Prior experience with data annotation, model evaluation, or writing guidelines for other reviewers is a plus.
  • Excitement to work in an early-stage environment on-site in San Francisco.

Why Join Collate?

Impact: The documents you help get right are the ones that put drugs, diagnostics, and devices in front of patients sooner.

Ownership: You define what "correct" means for a new category of AI product and own the data that enforces it, from day one.

Learning: Work daily with AI researchers, engineers, and life-sciences leaders; see how your labels change model behavior within the week.

Upside: Join early enough for meaningful equity and a role that grows with the data function—into leading a team of reviewers and contract SMEs as we scale.

Similar open jobs

Jobs in San Francisco (11750)