State Street is looking for a Lead Cloud Data Engineer to design, build, and optimize scalable data solutions on cloud platforms. You will leverage your industry experience, ideally in Financial Services, to build a robust cloud-based database engineering platform with strong technical expertise in Databricks and Spark.
What You'll Do
- Design, develop, and optimize end‑to‑end data pipelines using Databricks
- Implement and manage Lakehouse architecture with Delta tables
- Develop high‑performance PySpark and SQL transformations
- Ensure data quality, integrity, and performance across pipelines
- Lead troubleshooting, root cause analysis, and performance tuning
- Collaborate with Architects, Business Analysts, and downstream consumers
- Support CI/CD implementation for Databricks jobs and notebooks
- Mentor junior engineers and review code for quality and best practices
- Ensure adherence to security, governance, and compliance standards
What We're Looking For
- 10-14 years of hands-on experience with AWS Cloud Data Engineering on the Databricks platform
- Expertise in Data Modeling and Database design
- Hands-on experience in setting up data pipelines for ETL jobs across multiple data sources
- Hands‑on experience with performance tuning, Git, Jenkins, and enterprise job scheduling tools like Autosys or Airflow
- Basic understanding of Data warehousing and ETL concepts
- Knowledge of CI/CD tools like GitHub or Azure DevOps
- Advanced SQL and Python skills
- Experience working in Agile delivery environments
Nice to Have
- Experience in regulatory or financial services domain
- Databricks or Cloud certifications
- Experience with visualization tools like PowerBI or Tableau
- Exposure to Starburst, Spotfire, TDV, or ServiceNow
Technical Stack
- AWS, Databricks, Spark, PySpark, SQL, Delta tables
- Git, Jenkins, Autosys, Airflow
- GitHub, Azure DevOps, Python
- PowerBI, Tableau, Starburst, Spotfire, TDV, ServiceNow
Team & Environment
You will collaborate with Architects, Business Analysts, and downstream consumers in a culture that values technical leadership, mentorship, strong ownership, and delivery focus. We emphasize collaboration across global teams, proactive identification of risks and improvements, and we truly believe in the power that comes from diverse backgrounds and experiences.
Work Mode
This is a hybrid position based in Hyderabad or Bangalore.
As an Equal Opportunity Employer, we consider all qualified applicants for all positions without regard to race, creed, color, religion, national origin, ancestry, ethnicity, age, disability, genetic information, sex, sexual orientation, gender identity or expression, citizenship, marital status, domestic partnership or civil union status, familial status, military and veteran status, and other characteristics protected by applicable law.
