Cypher Consulting Europe is hiring an Azure Databricks Data Engineer

We are looking for an experienced Azure Databricks Data Engineer to help build and operate scalable, production-grade data platforms on Azure Databricks and Microsoft Azure. The ideal candidate combines strong Spark engineering expertise with a production engineering mindset, treating data pipelines as reliable, enterprise-grade products rather than ad-hoc solutions. This role requires hands-on experience delivering secure, governed, high-performance batch and streaming data platforms at scale. Tasks - Design, build, and maintain scalable batch and streaming pipelines using bronze/silver/gold architectures - Develop reusable transformation frameworks and production-grade Spark workloads - Implement and optimize Delta Lake solutions, including: ACID transactions, Partitioning strategies, Schema evolution, OPTIMIZE/ZORDER, Incremental and CDC processing patterns - Configure and manage Unity Catalog permissions, governance, lineage, and auditability - Perform Spark performance tuning and observability monitoring - Integrate Databricks with Azure services including ADLS, ADF/Synapse, Key Vault, and Microsoft Entra ID - Implement CI/CD pipelines and deployment automation using Azure DevOps or GitHub Actions - Collaborate with architects, analysts, data scientists, and engineers to align technical implementation with business needs - Continuously improve platform performance, reliability, cost efficiency, and standardization Requirements - Strong hands-on expertise with Azure Databricks engineering - Advanced Apache Spark experience using: - PySpark or Scala - Spark SQL - Deep understanding of Delta Lake architecture and optimization techniques - Experience building scalable enterprise data platforms and real-time pipelines - Strong governance and security experience in multi-team environments - Experience with service principals, managed identities, lineage, and auditability - Strong troubleshooting and Spark performance optimization skills - Azure integration experience across storage, orchestration, and security services - Solid CI/CD and Infrastructure-as-Code exposure - Comfortable working independently in consulting-style delivery environments