Informa Group Plc. is looking for a Lead Data Engineer to orchestrate and uphold high standards in Data Engineering and Enterprise Data Warehouse concepts throughout the Software Development Life Cycle. You will collaborate closely with development, operations, and architecture teams as part of an Agile engineering team, translating business needs into robust technical solutions.
What You'll Do
- Define delivery phases and perform project, task, and individual management duties for the team.
- Manage and enhance Continuous Integration, Delivery, and Deployment processes.
- Collaborate with stakeholders to produce accurate delivery estimates and oversee the transition from analysis to delivery.
- Design and implement scalable data solutions and data models for warehouse solutions using AWS services.
- Create and maintain Infrastructure as Code using CloudFormation.
- Implement comprehensive unit testing strategies.
- Work closely with cross-functional teams to gather requirements.
- Mentor junior engineers on AWS best practices and data engineering concepts.
- Participate in code reviews and technical discussions.
- Document technical solutions and maintain a knowledge base.
- Deploy, troubleshoot solutions, and plan/develop cloud-based applications and platforms.
- Ensure adherence to company security guidelines.
What We're Looking For
- Proven experience as a Lead Data Engineer with expertise in Python and Spark.
- Experience with CI/CD pipelines using GitHub and GitHub Actions.
- Experience developing and maintaining ETL pipelines using AWS Glue, Lambda, and EventBridge.
- Experience managing data storage solutions across S3 and Redshift.
- Experience implementing security best practices using AWS IAM.
- Hands-on experience with cloud services: AWS ECS, RDS, EFS, S3, Lambda.
- Experience with database management including SQL (MySQL) and NoSQL (MongoDB).
- Experience ensuring code quality through SonarCloud integration.
- In-depth knowledge of Test Driven Development (TDD) for Data Pipelines and Python.
- Excellent container orchestration skills using Docker.
- Strong understanding of Agile methodologies and implementations (Kanban, XP, Scrum).
- Excellent problem-solving, troubleshooting skills, and process orientation with great documentation skills.
- Must have the right to work and live in Bengaluru, India.
Nice to Have
- Bachelor’s degree in computer science or a related field, or equivalent work experience.
Technical Stack
- Python, Spark
- AWS Glue, AWS Lambda, AWS EventBridge, S3, Redshift, AWS IAM, AWS ECS, AWS RDS, AWS EFS
- SQL, MySQL, NoSQL, MongoDB
- SonarCloud, CloudFormation, Docker
- GitHub, GitHub Actions
Team & Environment
You will be part of an Agile engineering team, collaborating closely with development, operations, and architecture teams.
Benefits & Compensation
- 24 days annual leave
- 4 volunteering days annually
- Day off for your birthday
- Pension contributions
- Medical insurance for self and dependants
- Life cover and personal accident cover for self
- Seasonal social and charitable events
- Training and development
- Blended style and flexi working time
- Right tools for remote working
Work Mode
This role follows a hybrid work model and is based in Bengaluru, India.
Taylor & Francis is proud to be an Equal Opportunity Employer. We believe in and value diversity of people and thought, fostering a supportive and inclusive environment where all colleagues can learn and succeed as their true selves.
