Responsibilities
- Establish solution-level architectural frameworks with emphasis on Azure Databricks components including workspaces, compute strategies, and workflow orchestration.
- Design data mesh architectures and define domain-specific data products to support decentralized ownership and scalability.
- Implement Unity Catalog for centralized governance, security policies, data lineage tracking, and cross-workspace management.
- Help shape the long-term architectural roadmap and execution approach for data programs, ensuring consistency with organizational goals and platform guidelines.
- Create high-level system designs and lead their progression into actionable implementation plans in collaboration with system architects and development teams.
- Document and manage interdependencies across systems, integration touchpoints, and potential architectural risks.
- Architect efficient data pipelines for ingestion, transformation, and serving within Databricks environments.
- Define data product contracts, ownership models, and lifecycle processes to ensure operational clarity and sustainability.
- Implement secure access controls using Unity Catalog, Entra ID, and native cloud security mechanisms.
- Ensure all technical solutions comply with enterprise-wide governance, regulatory, and cybersecurity requirements in coordination with central architecture and platform groups.
- Support the continuous improvement of reference architectures, foundational capabilities, and reusable data design patterns across data services.
- Lead proof-of-concept initiatives to validate design choices, mitigate delivery risks, and assess emerging technologies.
- Maintain active involvement in coding tasks when needed, using Python for data engineering workflows, notebook development, and platform automation.
- Utilize C# and Python to build service integrations, APIs, and components that interface with the broader data platform.
- Assist implementation teams by clarifying architectural direction, evaluating technical designs, and resolving complex obstacles.
- Work closely with Product Management and business stakeholders to define epics and features during planning cycles and program increments.
- Coordinate with System Architects to maintain alignment between solution and system-level architectural decisions.
- Guide Agile delivery teams through key architectural decisions from planning through production deployment.
- Take part in formal architecture reviews, Inspect & Adapt sessions, and iterative solution improvements.
- Mentor System Architects, Technical Leads, and senior engineers on sound architectural practices.
- Advocate for industry best practices in data architecture, engineering workflows, and platform design principles.
- Facilitate knowledge sharing through design critiques, technical workshops, and comprehensive documentation.
- Produce C4 model diagrams using diagram-as-code tools such as Structurizr, and create sequence diagrams with PlantUML or Mermaid.
- Contribute to High-Level Designs and translate them into detailed Low-Level Designs, including data flows, class models, and interaction diagrams.
- Lead the identification and delivery of architectural enabler features from initial concept through final implementation.
