Ciklum is looking for a Senior Data Engineer to join a cross-functional team building high-speed data highways for telemetry and logs. This role is central to transforming diverse telemetry into a unified, AI-ready streaming architecture.
What You'll Do
- Build, deploy, and maintain mission-critical analytics solutions that process log and metric data at scale.
- Contribute design, code, and configurations for real-time streaming and data ingestion via Apache Kafka.
- Develop data pipelines to ensure total end-to-end responsibility of data flow from OTel Gateways to Loki and Prometheus.
- Perform performance tuning of complicated data flows and stream processing logic.
- Identify gaps and improve the platform’s quality, robustness, and speed for the Virtual Engineer AI consumer.
What We're Looking For
- 3+ years of experience coding in Python with solid CS fundamentals.
- 2+ years contributing to production deployments of large backend data processing systems.
- 1+ years of hands-on implementation experience with Kafka, Spark, or similar streaming technologies.
- 2+ years of experience in cloud data platforms, specifically Azure.
- Knowledge of Data Quality testing, automation, and telemetry visualization.
- Experience designing and documenting key components in large distributed computing systems.
Technical Stack
- Python
- Apache Kafka
- Spark
- Azure
- OpenTelemetry
- Loki
- Prometheus
Team & Environment
You will work within a cross-functional development team.
Benefits & Compensation
- Work alongside top professionals in a friendly, open-door environment.
- Take on large-scale projects with a global impact and expand your expertise.
- Boost your skills with internal events (meetups, conferences, workshops), Udemy access, language courses, and company-paid certifications.
- Explore diverse domains through internal mobility.
- Company-paid medical insurance, mental health support, and financial & legal consultations.
Work Mode
This position is local-country and based in India.
Ciklum is an equal opportunity employer.
