About the Role
Design and implement enterprise-grade data platforms on Azure Databricks, ensuring scalability, reliability, and performance. Lead technical architecture decisions, support data integration, and collaborate with engineering and analytics teams to deliver robust data pipelines and analytics capabilities.
Responsibilities
- Develop and maintain scalable data architectures on the Azure Databricks platform
- Lead the design of data models and data pipelines for large-scale analytics
- Ensure platform reliability, performance, and security standards are met
- Collaborate with data engineers and data scientists to implement analytics workflows
- Optimize data processing frameworks for efficiency and cost-effectiveness
- Integrate data from multiple sources into unified data environments
- Support the deployment and configuration of Databricks clusters and workspaces
- Define and enforce data governance and metadata management practices
- Troubleshoot and resolve technical issues within the data platform
- Evaluate and implement new tools and technologies to enhance platform capabilities
- Document system architecture and technical specifications
- Mentor team members on best practices in data platform development
- Ensure compliance with data privacy and security policies
- Work closely with cloud infrastructure teams to manage resource provisioning
- Drive automation of deployment and monitoring processes
- Participate in capacity planning and system scalability assessments
- Support disaster recovery and backup strategies for data platforms
- Contribute to technical roadmaps and strategic planning sessions
- Facilitate knowledge transfer across teams and stakeholders
- Monitor platform usage and provide optimization recommendations
Compensation
Competitive salary and benefits package
Work Arrangement
Hybrid working model with flexibility for remote and office-based work
Team
Collaborative data engineering and analytics team focused on delivering advanced data solutions
Technology Stack
- Primary platform: Azure Databricks
- Cloud infrastructure: Microsoft Azure
- Data storage: Azure Data Lake Storage, Delta Lake
- Orchestration: Azure Data Factory, Airflow
- Languages: Python, Scala, SQL
- DevOps: Azure DevOps, GitHub Actions
- Monitoring: Azure Monitor, Log Analytics
Professional Development
- Access to training programs and certification sponsorships
- Regular technical workshops and knowledge-sharing sessions
- Opportunities to attend industry conferences
- Internal innovation labs for emerging technologies
- Mentorship programs for career advancement
Available for qualified candidates requiring sponsorship
