LanceDB is hiring an Open Source Engineer to advance high-performance multimodal databases. You will leverage Java/Scala and Rust to expand the reach of Lance and LanceDB within the broader data infrastructure ecosystem.

What You'll Do

Drive OSS community efforts to integrate Lance format into Spark, Hive Metadata Store, Presto, Trino, Ray, and other data infrastructure systems.
Promote the Lance format at big data conferences and meetups.
Design and maintain efficient distributed Lance dataset operations.
Design efficient indices to power predicate pushdown in Spark, Ray, or Trino.
Work on table format, data encodings, and various aspects of the Lance format in Rust.
Operate on in-house data processing infrastructure.

What We're Looking For

At least five years of experience building high-performance databases, big data systems, or web-scale data services.
Experience with internals of open source big data or AI training systems, such as Hadoop, Spark, Flink, Ray, Iceberg, Delta-lake, Hudi, Clickhouse, Trino, Presto, PyTorch, or JAX.
Hands-on experience with high-performance computing in Java or Scala.
You thrive in a small, high-caliber team with autonomy, drive, and the ability to iterate fast.

Nice to Have

You are an open-source veteran, committer, or PMC of large open source systems in the Apache community.
You fearlessly challenge the status quo and dismiss mediocre engineering as unacceptable.
You have a proven record of driving large features in Apache projects.
You are familiar with Java, Rust, C++, Apache Arrow, Apache DataFusion, Apache Parquet, Apache Iceberg, and Delta Lake.

Technical Stack

Languages: Java, Scala, Rust, C++
Big Data Systems: Hadoop, Spark, Flink, Ray, Iceberg, Delta-lake, Hudi, Clickhouse, Trino, Presto
AI Frameworks: PyTorch, JAX
Apache Ecosystem: Apache Arrow, Apache DataFusion, Apache Parquet