Responsibilities
- Build and deploy data ingestion pipelines that connect with internal and third-party systems.
- Create reusable modules for data processing, integrity checks, and system logging.
- Work on both batch and real-time data flows, focusing on long-term scalability and ease of maintenance.
- Enhance system visibility through improved monitoring, alerting, and fault recovery mechanisms.
- Engage in technical design sessions, peer code evaluations, and root cause analyses for incidents.
- Collaborate with data users to gather needs and design effective ingestion strategies.
- Strengthen automation and testing practices to minimize manual intervention and improve system dependability.
