Responsibilities
- Create and manage cloud-based data pipelines that handle both batch and near-real-time processing needs.
- Build and streamline data workflows for ingestion, transformation, and orchestration using Azure Data Factory, Microsoft Fabric, Databricks, and similar tools.
- Deploy and support lakehouse and medallion-style data architectures with layered zones for analytics and AI use cases.
- Write efficient transformation logic in SQL, PySpark, and Python, emphasizing code performance and long-term maintainability.
- Work closely with analytics and business intelligence engineers to prepare data for semantic modeling and AI integration.
- Support data science teams by enabling feature engineering and model training pipelines.
- Establish data validation, quality checks, lineage tracking, and monitoring to ensure reliability of data assets.
- Define and apply data security standards, including encryption, access permissions, and data classification policies.
- Automate infrastructure provisioning and deployment processes using DataOps and infrastructure-as-code methods.
- Monitor and improve cloud resource efficiency to control costs and enhance system performance.
- Document technical designs, operational procedures, and architectural diagrams for knowledge sharing and continuity.
- Offer technical guidance and mentorship to less experienced data engineering team members.
Compensation
Competitive salary and benefits package offered based on experience and qualifications.
Work Arrangement
Hybrid work model with a combination of remote and on-site presence.
Team
Part of the Corporate Data Analytics Group focused on enterprise-wide data solutions.
Qualifications
- Bachelor’s degree in Computer Science, Engineering, or a related technical field.
- Minimum of 5 years of experience in data engineering with a focus on cloud platforms.
- Proven experience with Azure data services including Data Factory, Databricks, and Microsoft Fabric.
- Strong coding skills in SQL, Python, and PySpark for data transformation and pipeline development.
- Experience building and managing data lakehouse architectures with medallion zone patterns.
- Familiarity with DataOps, CI/CD, and infrastructure-as-code tools such as Terraform or ARM templates.
- Knowledge of data security principles including encryption, role-based access control, and data masking.
- Ability to collaborate effectively with data scientists, analysts, and BI developers.
- Experience with data observability, lineage tools, and data quality frameworks.
- Strong problem-solving skills and attention to detail in complex data environments.
Preferred Qualifications
- Master’s degree in Computer Science, Data Science, or related discipline.
- Experience with AI/ML workflows and supporting data infrastructure.
- Contributions to open-source data projects or active data community involvement.
- Certifications in Microsoft Azure data and analytics solutions.
Not available for this position.
