Lead Digital Data Engineer (Databricks)

BJ's Wholesale Club, Inc.

Where
BJ's Club Support Center Marlborough, MA #5997
Pay
$129K a year

Posted 23 days ago· Checked live at BJ's Wholesale Club, Inc. 4 days ago

Stale: open 23 daysUp for more than two weeks. It may already be filled.

See how you score for this job

One sentence and your resume. We score it against what BJ's Wholesale Club, Inc. asks for, line by line.

See my score
Is this job still open?
Yes. We checked it live on BJ's Wholesale Club, Inc.'s own hiring page 4 days ago.
What is the job?
A World-Class Team BJ’s Wholesale Club is powered by more than 30,000 team members who make a real impact every day. Whether you're stocking shelves, solving problems or shaping strategy, your work helps families save on what matters most.
What does it pay?
$129K a year, as the posting states.
Our verdict
Stale. Up for more than two weeks. It may already be filled.

What the job is

A World-Class Team

BJ’s Wholesale Club is powered by more than 30,000 team members who make a real impact every day. Whether you're stocking shelves, solving problems or shaping strategy, your work helps families save on what matters most.

We’re a team built on purpose and opportunity. Join us and be part of something meaningful.

Why You’ll Love Working at BJ’s

Key Responsibilities

  • Design and maintain a scalable Lakehouse using Delta Lake and the medallion (Bronze/Silver/Gold) architecture, serving as the single source of truth for analytics following the migration from AWS Redshift.
  • Build batch and real-time pipelines integrating Kafka, DynamoDB, S3, and other AWS services with Databricks (Structured Streaming, Auto Loader, Delta Live Tables).
  • Ingest and integrate digital sources such as Adobe Analytics, AEP, and Google Analytics alongside transactional and operational data.
  • Implement data quality validation frameworks with automated checks, alerting, and SLAs so data is accurate and clearly represented across business verticals.
  • Lead data governance through Unity Catalog (access control, lineage, classification) and monitor pipeline health, freshness, and cost.
  • Design reusable Genie Spaces and AI/BI dashboards with certified metrics for business self-service.
  • Develop and productionize Databricks agents and AI use cases using Mosaic AI, MLflow, Model Serving, and Vector Search.
  • Optimize performance and cost across clusters, SQL Warehouses, and Delta tables.
  • Establish CI/CD for data pipelines (Databricks Asset Bundles, Git workflows, automated testing, Terraform).
  • Own and prioritize the digital data and analytics roadmap with product leaders.
  • Lead and mentor a distributed team of up to 10 engineers across onshore and offshore locations, driving Agile delivery and providing regular status and risk updates to senior leadership.

Minimum Requirements

Bachelor's degree in Computer Science, Information Technology, or equivalent, plus 7+ years of experience in data engineering, including 3+ years of hands-on Databricks experience in production. Experience must include:

  • Expert-level Apache Spark (PySpark and SQL), Delta Lake, and Lakehouse architecture
  • Unity Catalog, Databricks Workflows, and SQL Warehouses
  • Integrating AWS services: S3, Kafka (MSK/Confluent), DynamoDB, Lambda, and IAM
  • Building data quality and validation frameworks
  • Genie Spaces, AI/BI dashboards, and building Databricks agents or GenAI solutions (Mosaic AI, MLflow)
  • Building large-scale distributed data systems on AWS, Azure, or GCP
  • Working in an Agile Scrum environment and leading distributed teams
  • SDLC, CI/CD, version control, and test automation for data pipelines

This is a hybrid role. Tuesday through Thursday are in-office days at BJ's Club Support Center in Marlborough, MA and Monday and Friday are remote days.

In accordance with the Pay Transparency requirements, the following represents a good faith estimate of the compensation range for this position. At BJ’s Wholesale Club, we carefully consider a wide range of non-discriminatory factors when determining salary. Actual salaries will vary depending on factors including but not limited to location, education, experience, and qualifications. The pay range for this position is $128,500.00 - $164,000.00

We recognize the growing role of AI tools, including ChatGPT, and value familiarity with them. That said, we want to hear from your authentic self. Your application should reflect your own skills, experiences, and insights rather than AI-generated responses.