Responsibilities
- Design, build, and operate reliable, secure, and observable data pipelines and curated datasets that power enterprise reporting, analytics, and AI/ML use cases.
- Own engineering quality, performance, and cost optimisation — implementing robust data quality controls, testing frameworks, monitoring, and observability across the platform.
- Build and maintain production-grade data infrastructure on Azure / Microsoft Fabric, including data lakes, lakehouses, and modern data warehouse patterns.
- Produce feature-ready datasets and optimised data products that enable data scientists, AI engineers, and analytics teams.
- Lead AI/ML engineering use cases — applying engineering best practice to model pipelines, data preparation, and AI-ready dataset design at scale.
- Evaluate and adopt emerging data and AI engineering tools and patterns; drive continuous improvement of RES's data ecosystem.
- Define and implement CI/CD pipelines for data engineering workflows; apply infrastructure-as-code and automated quality gates as standard practice.
- Lead engineering automation to reduce manual effort, improve reliability, and accelerate time-to-insight.
- Apply containerisation and orchestration tooling (e.g. Docker, Airflow, or equivalent) to production data workflows.
- Drive architectural decisions and shape the direction of the data platform.
- Partner across architecture, governance, data modelling, and reporting to deliver coherent, end-to-end data and AI products.
- Mentor and support engineers; set the standard for quality, craft, and engineering rigour across the team.
Requirements
- Degree in computer science, data engineering, software engineering, or a related field — or equivalent hands-on experience.
- Significant experience (typically 7+ years) delivering enterprise-grade data engineering solutions in production environments.
- Proven track record as a Senior Data Engineer, including building large-scale data systems using modern approaches and making architectural decisions.
- Deep expertise in the Microsoft Azure data ecosystem — ADF, Synapse, Fabric, Purview, Unity Catalogue.
- Advanced Python skills including open-source data libraries, frameworks, and messaging systems.
- Strong experience building and maintaining production data infrastructure for AI and ML consumption.
- Experience with MLOps practices: CI/CD for data pipelines, automated testing, and infrastructure as code.
Nice to Have
- Experience with modern data stack tooling — dbt, Airflow, Prefect, or equivalent orchestration and transformation frameworks.
- Exposure to working alongside data scientists and AI engineers in a shared platform model.
- Experience with automation tooling such as Power Automate, Power Platform, or equivalent.
- Relevant certifications in Microsoft Azure, data engineering, or AI/ML.
