First American is hiring a Senior Data Engineer – Data Platform & Databricks Expert to be the cornerstone of our big data initiatives. In this remote role, you will architect, build, and optimize our core data platform with Databricks, serving as the internal subject matter expert and driving strategy and governance.
What You'll Do
- Serve as the Databricks Subject Matter Expert, leading the adoption, optimization, and evangelization of Databricks and its ecosystem, including Delta Lake, Unity Catalog, MLflow, and Databricks SQL.
- Architect, build, and optimize big data solutions using Databricks and modern platforms such as Azure SQL, Synapse, and Snowflake.
- Drive our data strategy to align with business goals, lead platform governance, and ensure compliance with industry standards.
- Continuously optimize Databricks clusters, jobs, and workflows for performance, reliability, and cost-efficiency, applying similar principles to adjacent platforms.
- Lead the development of best practices for coding, data management, and platform operations, and mentor engineers, data scientists, and analysts.
- Stay current with industry trends and innovations, engaging in Databricks forums and conferences to share knowledge.
- Lead incident response efforts, troubleshoot complex issues, and implement long-term solutions.
What We're Looking For
- At least 7 years of experience in data engineering.
- A minimum of 3 years architecting and operating Databricks solutions in production environments.
- Expert-level proficiency in Databricks, including Delta Lake and Unity Catalog.
- Experience implementing and optimizing ETL/ELT on modern platforms like Azure SQL, Synapse, Databricks, and Snowflake.
- Demonstrated ability to integrate multiple data sources, including structured and unstructured data, APIs, and streams.
- Expertise in Delta Lake and Apache Iceberg formats for scalable batch and near real-time processing.
- Strong programming skills in Python and PySpark with experience developing production-grade applications and optimizing Spark jobs.
- Strong proficiency in SQL and modern data modeling methods, including medallion architecture and dimensional modeling.
- Capability to design efficient data structures for analytics.
- Demonstrated ability to lead teams and mentor junior engineers and data scientists.
- Experience leading migrations from legacy platforms to modern data platforms, including Databricks.
- A Bachelor's or Master's degree in Computer Science, Engineering, or a related field, or equivalent experience.
Nice to Have
- Experience building CI/CD pipelines for data platforms using Databricks CLI, Terraform for infrastructure as code, and Git-based workflows.
- Practical experience with machine learning workflows and MLOps practices using Databricks ML tools and MLflow.
- Databricks certifications, such as Databricks Certified Data Engineer Professional.
- Experience with real-time streaming technologies like Structured Streaming or Kafka.
- Contributions to Databricks forums, conferences, or other industry communities.
Technical Stack
- Databricks, Delta Lake, Unity Catalog, MLflow, Databricks SQL
- Azure SQL, Synapse, Snowflake
- Python, PySpark, SQL, Apache Iceberg
- Terraform, Git
Benefits & Compensation
- Annual compensation range: $126,100 - $168,100
- Comprehensive benefits package including medical, dental, vision, 401k, and PTO/paid sick leave.
- Employee stock purchase plan.
Work Mode
This is a fully remote position.
First American is an equal opportunity employer in every sense of the term. We will consider for employment all qualified applicants, including those with arrest or conviction records, in a manner consistent with applicable state and local laws.
