Responsibilities
- Design and manage scalable batch and streaming data pipelines using Spark or Flink
- Create and maintain integrations between workflow orchestrators, data lake formats, and cloud infrastructure
- Apply proven methods in data structure design, partition strategies, and query efficiency improvements
- Support containerized data processing workloads on Kubernetes and diagnose operational issues
- Enhance data workflows for better performance, lower costs, and higher dependability
- Set up monitoring systems, logging frameworks, and data integrity validation rules
- Engage in on-call rotations and assist with resolving production incidents
Work Arrangement
Remote — Lucknow
