▪ Own and govern the data ingestion and integration architecture for the enterprise data platform.
▪ Define and evolve scalable, reusable, and production-grade data interfaces across all source systems and business domains.
▪ Provide technical leadership and design authority for all data platform ingestion and integration components, ensuring engineering quality, scalability, and reliability.
▪ Establish and enforce software engineering and architecture standards across data platform development (CI/CD, testing, observability, design patterns).
▪ Act as the primary escalation point and technical authority for complex ingestion, integration, and data platform challenges.
▪ Guide and support a distributed team of software engineers and data engineers in building production-grade ingestion services and platform components.
▪ Collaborate closely with platform architecture, cybersecurity, infrastructure, and business IT teams to ensure secure, compliant, and sustainable system design.
▪ Drive consistency and reuse across data platform patterns, frameworks, and engineering practices.
Main Tasks:
▪ Define and evolve ingestion frameworks for batch, streaming, API-based, and event-driven architectures.
▪ Establish and govern standards for structured and unstructured data (JSON, CSV, XML), including schema evolution and compatibility strategies.
▪ Design and review production-grade ingestion and integration patterns, ensuring fault tolerance, observability, and performance.
▪ Define and enforce API design standards, reliability patterns, and contract management.
▪ Lead design reviews for high-risk or complex ingestion pipelines, focusing on scalability, security, and maintainability.
▪ Support engineers in implementing robust error handling, retry mechanisms, orchestration, and monitoring.
▪ Collaborate with system owners to define interface specifications and integration strategies.
▪ Align ingestion architecture with overall platform design, governance, and business requirements.
▪ Enable ingestion patterns for advanced use cases, including data science, machine learning, and AI/LLM integrations.
▪ Maintain and evolve reusable libraries, templates, and framework components to accelerate development.
▪ Promote engineering best practices, including code reviews, testing strategies (unit, integration), and CI/CD pipelines.
▪ Define and maintain architecture blueprints, design guidelines, and engineering standards.
▪ Create reusable architecture patterns for lakehouse-based data platform.
▪ Provide guidance on scalability strategies, resource utilization, and performance optimization.
▪ Support cost transparency, usage optimization, and efficient resource consumption across the platform.
Electronic City Rd, Phase II, Electronic City, Bengaluru, Karnataka 560100, India On-site Full-time