Senior System Test Engineer

Lancedb

Where
United States | Canada

Posted 37 days ago· Checked live at Lancedb 2 hours ago

Stale: open 37 daysUp for more than two weeks. It may already be filled.

See how you score for this job

One sentence and your resume. We score it against what Lancedb asks for, line by line.

See my score
Is this job still open?
Yes. We checked it live on Lancedb's own hiring page 2 hours ago.
What is the job?
AI advances at the speed of its research, and research moves at the speed of its data. LanceDB is the AI-native Multimodal Lakehouse: one system where a researcher curates petabytes of video, audio, and every signal derived from them with a few lines of Python, and the next training run starts as fast as the next idea.
What does it pay?
The posting doesn't say.
Our verdict
Stale. Up for more than two weeks. It may already be filled.

What the job is

AI advances at the speed of its research, and research moves at the speed of its data. LanceDB is the AI-native Multimodal Lakehouse: one system where a researcher curates petabytes of video, audio, and every signal derived from them with a few lines of Python, and the next training run starts as fast as the next idea.

What you'll do

  • System test strategy: Define and drive system-level testing across correctness, reliability, scalability, and performance, prioritizing the highest-risk behaviors in the platform.
  • Test frameworks and workloads: Build automated test frameworks and realistic workloads covering ingestion, concurrent reads and writes, indexing, querying, and dataset maintenance.
  • Data integrity and fault injection: Validate data integrity and documented consistency guarantees under concurrency, retries, partial failures, and recovery. Develop fault-injection and resilience tests for process crashes, network disruptions, storage errors, resource exhaustion, and interrupted operations.
  • Performance testing: Design load, stress, and long-running soak tests, and identify regressions in latency, throughput, resource usage, and search quality.
  • Upgrades and compatibility: Test upgrades, compatibility, and recovery scenarios to ensure the platform behaves predictably as software and workloads evolve.
  • Partner closely with engineers to investigate failures using logs, metrics, traces, and source code, reduce complex issues to minimal reproductions, and drive fixes and regression coverage.
  • Develop and continuously improve reproducible test environments, integrating reliable, actionable checks into CI and release workflows.
  • Use AI coding assistants and other AI tools to develop test harnesses, explore failure scenarios, analyze results, and accelerate debugging, independently verifying generated code and conclusions.

What we're looking for

  • 10+ years of experience in software engineering, system test engineering, or test infrastructure, with substantial experience building or validating distributed databases or large-scale data systems
  • Deep understanding of distributed databases, including consistency models, replication, partitioning, concurrency control, durability, and failure recovery
  • Proven track record of designing tests that verify system invariants and uncover data corruption, race conditions, and failures that emerge only under load or partial outages
  • Strong programming skills in Python and at least one systems language such as Rust, C++, Java, or Go, with the ability to build maintainable test infrastructure and debug product code
  • Hands-on experience with fault injection, performance testing, workload generation, and automated testing in distributed environments
  • Demonstrated ability to use AI tooling adeptly in daily engineering work, including coding assistants and agent-based workflows, with sound judgment about validating outputs
  • Experience with Linux, cloud infrastructure, containers, CI/CD, and observability tools
  • Ability to own ambiguous projects, prioritize risk, communicate findings clearly, and collaborate closely with engineers to drive issues to resolution

Nice to have

  • Experience testing database storage engines, query engines, vector search, or cloud object storage integrations
  • Familiarity with LanceDB, Lance, Apache Arrow, DataFusion, Parquet, or related data infrastructure
  • Experience with property-based testing, fuzzing, differential testing, or distributed consistency testing
  • Experience with Kubernetes, infrastructure as code, and large-scale test environment orchestration
  • Contributions to open-source databases, distributed systems, or testing frameworks