Senior Software Engineer, Data Engineering

Distributed Spectrum

Where
New York City

Posted 44 days ago· Checked live at Distributed Spectrum 28 minutes ago

Stale: open 44 daysUp for more than two weeks. It may already be filled.

See how you score for this job

One sentence and your resume. We score it against what Distributed Spectrum asks for, line by line.

See my score
Is this job still open?
Yes. We checked it live on Distributed Spectrum's own hiring page 28 minutes ago.
What is the job?
DS creates systems that power the next generation of radio spectrum intelligence. We collect radio data from all over the world, train neural networks to decipher it, and run them on the smallest chips we can.
What does it pay?
The posting doesn't say.
Our verdict
Stale. Up for more than two weeks. It may already be filled.

What the job is

DS creates systems that power the next generation of radio spectrum intelligence. We collect radio data from all over the world, train neural networks to decipher it, and run them on the smallest chips we can.

What you'll do

  • Architect and build our data lakehouse: ingestion pipelines, open table formats, partitioning and compaction strategies, cataloging, and governance.
  • Design and operate vector database infrastructure for embedding storage, similarity search, and retrieval at scale — choosing, tuning, and evolving the right systems for our workloads.
  • Build large-scale data management systems: lifecycle and retention, lineage, versioning of datasets for reproducible training, quality monitoring, and cost management across petabyte-class storage.
  • Design data platform capabilities that directly support ML use cases — feature and embedding pipelines, training-set assembly, evaluation datasets, and low-latency retrieval for agents.
  • Establish schema and data-contract standards across teams, and build tooling that lets researchers and engineers self-serve.
  • Own reliability and performance of data pipelines in production, including observability and failure handling.
  • Mentor engineers and lead technical design across the data domain.

What we're looking for

  • 4–6+ years of software / data engineering experience, including several years designing and operating large-scale data platforms in production.
  • Deep experience with lakehouse architectures and technologies — e.g., Apache Iceberg, Delta Lake, or Hudi; Parquet; Spark, Flink, Trino, DuckDB, or similar engines.
  • Hands-on experience with vector databases and embedding retrieval (e.g., pgvector, Milvus, Qdrant, Weaviate, Pinecone, LanceDB, or FAISS-based systems), including indexing trade-offs and scaling.
  • Strong experience with AWS or another major cloud provider — object storage, managed data services, compute, and cost optimization at scale.
  • Fluency in Python and SQL; experience with orchestration tools (Airflow, Dagster, Prefect, Step Functions, etc.).
  • Demonstrated experience designing data architectures specifically for ML / AI workloads: training data pipelines, feature stores, dataset versioning, or retrieval systems.
  • Strong grasp of data modeling, consistency, and the trade-offs between batch and streaming.

Nice to have

  • Experience with streaming ingestion at high volume (Kafka, Kinesis, Pulsar).
  • Experience with time-series, geospatial, or signal / sensor data.
  • Familiarity with data governance and security requirements in regulated environments.
  • Experience with Rust, Go, or C++ for performance-critical data paths.

What we offer

  • Above-market salary, equity, and benefits package.
  • Early Series A Equity
  • Excellent health, dental, and vision coverage
  • 401(k) match - up to 4% of your salary
  • Flexible PTO
  • Daily office lunches in NYC

Itar requirements

To conform to U.S. Government technology export regulations, including the International Traffic in Arms Regulations (ITAR) you must be a U.S. citizen, lawful permanent resident of the U.S., protected individual as defined by 8 U.S.C. 1324b(a)(3), or eligible to obtain the required authorizations from the U.S. Department of State. Learn more about the ITAR here https://www.pmddtc.state.gov/?id=ddtc_kb_article_page&sys_id=24d528fddbfc930044f9ff621f961987.

Similar open jobs

Software engineer jobs in New York (1437)Jobs in New York (19157)