Responsibilities
- Design and build reliable, high-capacity data pipelines and data transformation systems
- Create and fine-tune data processing architectures using Apache Spark and Delta Lake
- Build and oversee data solutions within the Databricks environment
- Work closely with business and analytics groups to convert business needs into technical implementations
- Maintain high standards for data accuracy, consistency, speed, and protection across platforms
- Establish and uphold strong data engineering practices, coding standards, and governance policies
- Combine data from diverse internal and external systems using current integration technologies
- Advance CI/CD adoption and DevOps methods for data workflow automation
- Oversee, debug, and enhance data job performance and system efficiency
- Guide and support less experienced engineers with technical expertise
- Engage in system design evaluations and help shape technology architecture choices
