Responsibilities
- Leading large-scale data migrations from on-premises or legacy platforms to cloud-native architectures on AWS — including Lakehouse designs using Databricks or Snowflake
- Designing and building low-latency data pipelines that process real-time betting events, odds updates, and market data at high throughput
- Implementing robust data integrity controls — validation, reconciliation, and auditability — across betting, trading, and customer data domains
- Architecting streaming and micro-batch solutions using AWS Kinesis, Apache Kafka, or MSK for real-time sportsbook and in-play data feeds
- Working with AWS platform services including S3, Glue, Lambda, Athena, Redshift, Lake Formation, Step Functions, and MWAA
- Supporting regulatory and compliance reporting — ensuring data pipelines meet the auditability and lineage requirements of the UK Gambling Commission and international regulators
- Leading and mentoring a team of data engineers, setting standards for code quality, testing, and documentation
- Contributing to technical proposals, pre-sales scoping, and architecture reviews with iGaming clients
Requirements
- You know the iGaming data landscape - You've worked in or closely with sportsbook, exchange, or casino platforms — and understand the data models behind markets, selections, bets, and settlements
- You've built or maintained low-latency data feeds — odds ingestion, in-play event streams, price change feeds — where milliseconds matter
- You understand the data integrity challenges specific to betting: reconciliation between trading and settlement systems, detection of data drift, and auditability for regulatory purposes
- You've delivered or contributed to large-scale data migrations — moving high-volume transactional or event data from legacy estates to the cloud without losing fidelity or uptime
- You're strong on AWS data engineering - AWS is your primary cloud — you're confident across Glue, Lambda, Athena, S3, Redshift, RDS, Kinesis, MSK, and Step Functions
- You've built real-time or near-real-time pipelines using streaming frameworks — Kinesis Data Streams, Kafka, Flink, or Spark Streaming
- You understand AWS cost models and how to optimise high-throughput workloads — Spot compute, reserved capacity, and right-sizing for event-driven architectures
- Solid experience with Python, SQL, and Spark — and pipeline tools such as dbt or Airflow
- Comfortable with data quality frameworks and testing — Great Expectations, dbt tests, or custom validation layers
- You've led or mentored data engineers and are comfortable owning technical direction on a complex engagement
- You can navigate the politics of a large operator — multiple teams, competing priorities, and legacy systems that can't just be switched off
- You balance hands-on delivery with client engagement, and know when to push back on scope and when to be pragmatic
Nice to Have
- AWS certifications are a bonus — Solutions Architect or Data Analytics Specialty
- Experience with infrastructure-as-code (Terraform, AWS CDK) is a plus
- Familiar with broader cloud platforms — Azure (Data Factory, Synapse) or GCP (Dataflow, BigQuery) — is beneficial but not essential
Work Arrangement
Hybrid — Edinburgh, Leeds, Manchester, London, Bulgaria
Additional Information
- occasional travel to client sites or CreateFuture offices when needed