Apply on company website London, England, United Kingdom Hybrid Full-time

Methods Business and Digital Technology Limited is hiring a Mid/Senior Data Engineer

About the Role

You will design and implement data pipelines, develop data models, optimize processing systems, and collaborate with stakeholders to turn business needs into technical solutions, while contributing to data governance and engineering best practices.

Responsibilities

  • Create and enhance ETL and ELT workflows for ingesting, profiling, reconciling, cleaning, and reporting data from enterprise systems.
  • Develop data catalogs, flow diagrams, interface definitions, and trusted source datasets.
  • Design and implement modern data architectures aligned with business and technical goals, supporting both current and future-state designs.
  • Support clients in strengthening trust in operational, workforce, procurement, and financial reporting through reliable, timely data.
  • Construct scalable and high-performance data systems using cloud platforms and open-source tools.
  • Design data models to meet complex enterprise analytical demands.
  • Improve large-scale data processing systems for better performance and lower operational costs.
  • Establish comprehensive data quality frameworks and monitoring mechanisms.
  • Assess emerging technologies to advance data engineering capabilities.
  • Work with stakeholders to convert business requirements into actionable technical designs.
  • Communicate technical approaches clearly to leadership and non-technical audiences.
  • Support the growth of the Analytics Engineering Practice through active participation in internal knowledge-sharing communities.

Requirements

  • Experience extracting and processing data from systems such as Ariba, Workday, SAP S/4HANA, or similar sources including spreadsheets, procurement, finance, and workforce platforms.
  • Proven ability in identifying data quality issues, defining cleansing rules, mapping data across systems, validating reconciliation results, and documenting exceptions.
  • Skill in working iteratively with architects, process owners, and functional teams to transform unclear business problems into structured data engineering actions.
  • Familiarity with data ownership, stewardship, lineage, metadata management, and controls, with ability to create reusable governance documentation.
  • Experience applying test-driven development in data workflows, including unit and integration testing, and data validation frameworks.
  • Demonstrated leadership in technical delivery of data projects.
  • Strong expertise in data architecture and modeling for scalable solutions.
  • Solid understanding of data warehouse design principles and methodologies.
  • Advanced knowledge of performance optimization for large-scale data processing.
  • Proficiency in SQL and Python for solving complex data challenges.
  • Hands-on experience with Apache Spark, including PySpark or Spark SQL.
  • Experience using Azure data services and tools.
  • Familiarity with workflow orchestration platforms such as Azure Data Factory or Apache Airflow.
  • Experience using containerization tools like Docker.
  • Proficiency in dimensional modeling techniques.
  • Experience implementing CI/CD pipelines for data systems.
  • Strong communication skills for explaining technical concepts to diverse audiences.

Nice to Have

  • Experience designing or implementing data mesh or data fabric architectures.
  • Knowledge of cost optimization strategies for cloud-based data platforms.
  • Experience applying data quality frameworks and improvement initiatives.
  • Familiarity with data visualization tools such as Power BI or Apache Superset.
  • Experience with alternative cloud data platforms including AWS, GCP, or Oracle.
  • Hands-on work with modern unified data platforms like Databricks or Microsoft Fabric.
  • Experience using Kubernetes for container orchestration.
  • Understanding of streaming data technologies such as Apache Kafka and event-driven architectures.
  • Experience building high-performance, large-scale data systems.

Work Arrangement

Hybrid — London, Sheffield, Bristol

What You'll Be Doing as a Data Engineer

  • Design, build and improve ETL and ELT pipelines that support data ingestion, profiling, reconciliation, cleansing and reporting across enterprise source systems.
  • Building data catalogues, data flows, interface views and trusted source views
  • Design and architect modern data solutions that align with business objectives and technical requirements, supporting current-state and target-state data architecture
  • Help clients improve confidence in operational, workforce, procurement and financial reporting through timely, accurate and reconcilable data.
  • Build highly scalable and performant data solutions leveraging cloud platforms and open-source software
  • Develop data models to handle enterprise-level analytical needs
  • Optimise large-scale data processing systems for performance and cost-efficiency
  • Implement robust data quality frameworks and monitoring solutions
  • Evaluate new technologies to enhance our data engineering capabilities
  • Collaborate with stakeholders to translate business requirements into technical specifications
  • Present technical solutions to leadership and non-technical stakeholders
  • Contribute to the development of the Methods Analytics Engineering Practice by participating in our internal community of practice

Your Impact

  • Enable business leaders to make informed decisions with confidence through timely, accurate data insights
  • Establish reusable engineering standards, patterns and documentation that support quality, maintainability and repeatable Data Foundations delivery across future engagements.
  • Drive adoption of modern data architectures and platforms
  • Deliver seamless data solutions that enhance user experience
  • Elevate the technical capabilities of the entire data engineering team
  • Help cultivate a data-driven culture within the organisation
  • Establish technical standards and patterns that ensure quality and maintainability

Other

  • UKSV (United Kingdom Security Vetting) clearance is required for this role, with Security Check (SC) as the minimum standard, either already held or with a willingness to undergo the process.
  • Some roles/projects may require Developed Vetting (DV) clearance; while not mandatory, a willingness to obtain DV clearance would be beneficial.
  • As part of the onboarding process candidates will be asked to complete a Baseline Personnel Security Standard (BPSS); details of the evidence required to apply may be found on the government website GOV.UK – Government baseline personnel security standard.
  • If you are unable to meet this and any associated criteria, then your employment may be delayed, or rejected.
  • Details of this will be discussed with you at interview.
  • The role is mainly remote but requires flexibility to travel to client sites and offices in London, Sheffield, and Bristol.
Required Skills
SQLApache SparkGoogle Cloud PlatformOracle
Test your skills for this role

Take a short quiz and show this employer what you can do.

Job Details
Location London, England, United Kingdom
Work mode Hybrid
Employment Full-time
Department Data and AI
Category Data & ML
Posted 21 days ago
Application On company website
or drop your CV first
About company
Methods Business and Digital Technology Limited logo
Methods is a £100M+ IT Services Consultancy established over 30 years ago, UK-based, applying skills in transformation, delivery, and collaboration to create end-to-end business and technical solutions for the public sector and a growing private sector portfolio. It was acquired by the Alten Group in early 2022.
All jobs at Methods Business and Digital Technology Limited Visit website