Canada Life UK is looking for a Lead Data Engineer to provide hands-on technical leadership within our Enterprise Data Platform. In this role, you will leverage your expertise in Azure and Databricks to build, deploy, and support high-performance data-driven solutions.

What You'll Do

Interpret Outcomes and user stories, translating them into technical solutions.
Create innovative solution designs for domain and enterprise data products.
Lead Product Increment planning to break down solutions into Features and Epics for incremental delivery.
Design and implement scalable data solutions on Azure and Databricks within your assigned domain.
Ensure appropriate engineering standards are applied to maintain data quality, performance, and reliability.
Work with Product Owners and Business Analysts to understand Outcomes and refine user stories.
Lead solution design for Finance and Enterprise data products, ensuring alignment with enterprise patterns.
Direct and collaborate with Data Analysts on detailed data discovery and requirements refinement.
Oversee logical and physical data modelling for Finance and Enterprise data products.
Implement and maintain data pipelines and ETL workflows in Databricks using PySpark and Delta Lake.
Contribute to CI/CD pipelines for data applications using Azure DevOps and infrastructure-as-code (Terraform).
Apply security, access control, and compliance standards for Azure and Databricks.
Support monitoring, logging, and basic cost optimisation for the team’s data products.
Support the development of DevOps practices within the team, including reducing technical debt and improving automation.

What We're Looking For

A degree-level IT or technical/scientific subject qualification, or equivalent experience.
Strong experience working within an agile development methodology, ideally Scaled Agile (SAFe or similar).
Proven and demonstrable data engineering capability.
Strong experience with Databricks (Spark, PySpark, Delta Lake, and Unity Catalog advantageous).
Proficiency in Azure data services (Azure Data Factory, Data Lake, Azure Functions advantageous).
Experience contributing to CI/CD pipelines (Azure DevOps, GitHub Actions, Terraform).
Scripting and programming skills (Python advantageous).
Good understanding of DevOps and automation concepts (e.g., YAML pipelines, IaC).
Solid understanding of cloud security, compliance, and governance principles.
Experience working with Databricks and Azure in a product or Scaled Agile delivery environment.
Automation experience, including testing of data pipelines and data products.
Strong teamwork, communication, and problem-solving skills to collaborate effectively with cross-functional teams.
Awareness of security principles and best practices to ensure secure data solutions.
Commitment to continuous learning and staying current with Azure, Databricks, and data engineering trends.
Excellent time and self-management through effective planning and prioritisation of tasks.
Ability to influence within the team and communicate clearly with technical and non-technical stakeholders.

Nice to Have

Microsoft Azure Data Engineer or Solutions Architect certification.
Databricks Certified Data Engineer or Machine Learning Associate certification.
Experience with streaming solutions (Kafka, Event Hubs, Spark Streaming).
Knowledge of machine learning and AI on Databricks.