Senior Manager AI Model Validation

Company 19 - John Hancock Life Insurance Company (U.S.A.)

Where
Toronto, Ontario
Pay
$129K a year

Posted 14 days ago· Checked live at Company 19 - John Hancock Life Insurance Company (U.S.A.) 4 days ago

Live.Open on the employer's own site.

See how you score for this job

One sentence and your resume. We score it against what Company 19 - John Hancock Life Insurance Company (U.S.A.) asks for, line by line.

See my score
Is this job still open?
Yes. We checked it live on Company 19 - John Hancock Life Insurance Company (U.S.A.)'s own hiring page 4 days ago.
What is the job?
This is a high-visibility, high-impact role suitable for experienced data scientists who also have strong communications ability and business awareness, as well as the ability to make judgment calls in ambiguous situations. You will support Global AI Model Validation and Governance’s mandate of identifying, assessing, and communicating AI/ML model risks across all AI/ML models deployed across Manulife globally.
What does it pay?
$129K a year, as the posting states.
Our verdict
Live. Open on the employer's own site.

What the job is

This is a high-visibility, high-impact role suitable for experienced data scientists who also have strong communications ability and business awareness, as well as the ability to make judgment calls in ambiguous situations. You will support Global AI Model Validation and Governance’s mandate of identifying, assessing, and communicating AI/ML model risks across all AI/ML models deployed across Manulife globally.

Position Responsibilities

  • Provide independent challenge and constructive feedback for AI/ML, GenAI, and agentic models by identifying model weaknesses, limitations, assumptions, and risks, and recommending appropriate remediation actions.
  • Manage competing deadlines, competing expectations, and coordinate procedural alignment to ensure the balance of risk management activities is commensurate with the business risk exposure.
  • Communicate complex technical concepts and validation findings to technical and non-technical stakeholders, while ensuring that the risk management process is followed.
  • Review model documentation, assess model design and performance, and communicate validation findings through clear, evidence-based reports and recommendations.
  • Conduct model testing, benchmarking, sensitivity analysis, and other validation activities.
  • When appropriate, evaluate whether AI models are reliable, fair, safe, explainable, and fit for their intended purpose.
  • Support model inventory management, model risk operations and other AI governance processes.
  • Mentor and support junior data scientists through technical coaching, knowledge sharing, and review of validation approaches and methodologies.

Required Qualifications

  • Master's degree or higher in Statistics, Biostatistics, Data Science, Engineering, Mathematics, or a related quantitative field.
  • At least 3-5 years of a mix of hands-on data science experience (data science, statistics, AI/ML model development, AI/ML research, model validation, model risk management, or a related quantitative discipline) and a stakeholder-exposed role (functions that interact with varying teams and business units across the organization.)
  • Strong critical thinking skills and the ability to independently challenge assumptions and assess model risk in the context of the business problem it solves.
  • Hands-on keyboard report writing and code implementation; being able to implement your ideas yourself.
  • Experience interpreting model performance metrics and conducting quantitative analyses in a variety of standard and non-standard contexts.
  • Strong communication, written documentation, and stakeholder management skills.
  • The ability to collaborate with parties from all walks of a large corporation.

Preferred Qualifications

  • Prior model validation experience for AI/ML models or development of AI/ML models for production environments with tangible risk exposures. Comparable experience includes model development in stringently risk-controlled contexts such as pharmaceuticals, aerospace, etc.
  • Knowledge of AI governance and model risk regulatory expectations, with emphasis on OSFI E-23.
  • Familiarity with Responsible AI concepts, including fairness, explainability, robustness, transparency, safety, and monitoring. Being passionate about Responsible AI is a plus.
  • Experience evaluating Generative AI, LLMs, RAG systems, or agentic AI solutions built or customized to solve a very broad range of business problems.
  • Experience developing validation frameworks, testing libraries, or automated assurance tooling.
  • Experience working in a global, cross-functional environment.
  • Experience performing research on AI/ML models in academic or industry settings.

Referenced Salary Location

Toronto, Ontario

Working Arrangement

Hybrid

Salary range is expected to be between

Similar open jobs

Jobs in Boston (7107)Jobs in Toronto (831)