Responsibilities
- Ensure availability, stability, observability, and scalability of the data platform.
- Own production issue resolution, RCA, and continuous improvements.
- Lead design and delivery of platform features.
- Make informed choices across data architectures (e.g., batch vs. streaming, Delta vs. Iceberg), orchestration tools (Airflow, Argo), and scalability patterns.
- Implement best practices for partitioning, indexing, and resource optimization.
- Drive cost efficiency through FinOps-aligned design and observability.
- Treat platform as a product—balance usability, governance, and adoption.
- Define and track KPIs for usage, performance, and cost.
- Write modular, testable, and maintainable code.
- Build CI/CD pipelines, structured logging, and enable observability and debugging by design.
- Collaborate across teams, own stakeholder communication, drive sprint execution, and contribute to documentation.
- Mentor engineers and influence best practices.
Requirements
- 7–10 years in platform/data engineering.
- Proficient in Python, Shell, Kubernetes, CI/CD, Terraform, observability tools, and Linux-based systems.
- Hands-on with Azure/GCP/AWS, modern orchestration tools, data formats (Parquet, Delta, Iceberg), and compute engines (Databricks, Clickhouse, Trino).
- Design scalable, fault-tolerant systems with efficient storage, caching, and cost control.
- Apply data access policies, SLA/SLO tracking, and schema evolution best practices.
- Enable monitoring, auditing, and incident management.
- Take initiative, prioritize business impact, mentor team members, and drive accountability in delivery and technical decision-making.
Nice to Have
- Experience with model serving frameworks is a plus.
Benefits
- Modern work environment, flexible schedule and smart, creative, down-to-earth people.
- Internal opportunities to move roles and try out bridge assignments with different teams.
- Free meals all days of the week.
- Gym or Yoga class.
- Cocktails at drink cart Thursdays.
- Fun at work on Funky Fridays.
- Bring your kids and pets to work.