Responsibilities
- Develop and maintain the data platform's infrastructure using Terraform.
- Implement and manage CI/CD pipelines for the automation of data and ai processes and pipelines.
- Automate the provisioning, configuration, and monitoring of resources in AWS.
- Apply GitOps principles to manage configurations and deployments.
- Ensure the security, scalability, and resilience of the infrastructure through automation.
- Collaborate with Data Engineers to optimize the data development lifecycle (DDLC).
- Own the automation and infrastructure of orchestration tools like Airflow.
- Document the automation architecture and deployment processes.
Requirements
- Solid experience with Infrastructure as Code (IaC) tools, especially Terraform.
- Experience in creating and managing CI/CD pipelines (e.g., Github Action, AWS CodePipeline).
- Deep knowledge of version control tools and workflows with GIT.
- Experience in automation scripting with Python, AWS CLI, Spark.
- Knowledge of AWS services automation and management.
- University Degree in Computer Science, Engineering or equivalent
- At least 2 years of experience in DataOps, DevOps, or infrastructure automation
- Excellent verbal and writing English communication skills
- Good learning mindset
- Able to set priorities and multitask
Nice to Have
- Familiarity with the AWS data ecosystem (S3, Glue, Lambda, Redshift, Snowflake).
- Familiarity with orchestrators like Airflow.
- Familiarity with data streaming technologies such as Kafka.
- Knowledge of monitoring and visualization tools (Prometheus, Grafana, CloudWatch).
- Basic knowledge of containerization (Docker, Kubernetes).
- Excellent verbal and written English communication skills.
Additional Information
- Excellent verbal and written English communication skills.
