Sagis Diagnostics is hiring a Data Engineer II to join our Data Architecture team. This role is central to evolving our data platform across cloud, on-premise, and hybrid environments. You'll design, build, and maintain complex data pipelines that serve critical clinical and business functions.
What You'll Do
- Design, build, and maintain scalable ETL/ELT pipelines that move data across disparate source systems including LIS (Laboratory Information Systems), billing platforms, and cloud data stores.
- Develop and optimize data workflows in Databricks using PySpark, Delta Lake, and Unity Catalog, following medallion architecture (Bronze → Silver → Gold) patterns.
- Architect and manage data integrations with Azure Data Factory, Azure SQL Managed Instance, and Azure Blob/ADLS Gen2 storage.
- Collaborate closely with the Director of Data Architecture to plan, execute, and document major pipeline and schema changes.
- Build and maintain robust Python-based automation scripts, data quality checks, and monitoring routines.
- Support reporting and analytics platforms including Power BI and Zoho Analytics, ensuring clean, performant data models and dataset refresh pipelines.
- Participate in schema design and maintenance for SQL Server environments including stored procedures, views, and schema-level organization.
- Contribute to data governance practices: access controls, Unity Catalog permissions, lineage documentation, and change management.
- Engage with internal stakeholders (clinical, billing, operations) to translate business requirements into data engineering solutions.
- Identify and resolve data quality, latency, and pipeline reliability issues proactively.
What We're Looking For
- A seasoned Data Engineer with deep hands-on experience designing, building, and maintaining complex ETL pipelines and data workflows.
- Proven experience with the core technical stack: Databricks, PySpark, Delta Lake, and Azure data services.
- Strong proficiency in Python and SQL for data engineering tasks.
- Experience with data modeling, schema design, and maintaining data pipelines for analytics platforms.
- Ability to collaborate effectively with technical leadership and cross-functional business stakeholders.
- A proactive approach to identifying and resolving data quality and pipeline reliability issues.
Technical Stack
- Primary: Databricks, PySpark, Delta Lake, Unity Catalog
- Azure: Data Factory, SQL Managed Instance, Blob/ADLS Gen2
- Database: SQL Server
- Language: Python
- Analytics: Power BI, Zoho Analytics
Team & Environment
You will join the Data Architecture team and report directly to the Director of Data Architecture. This role is a key contributor to our data platform evolution.
