Responsibilities
- Create and sustain comprehensive data pipelines and backend ingestion workflows, contributing to the development of the Data Platform for enhanced automation and analytics.
- Handle data from diverse origins such as ERP (Netsuite), CRM (Salesforce), product, order flow, and support ticket information.
- Oversee essential data pipelines to support growth projects and sophisticated analytics.
- Enable data integration and transformation for application data transfers, ensuring compatibility with data layers and the data lake.
- Enhance data architecture, quality, monitoring, observability, and accessibility.
- Code data transformations using SQL and Python to produce data products used by analytics, marketing operations, and sales operations groups.
- Construct and manage large-scale Spark and PySpark workflows for batch and streaming data processing in Databricks and cloud settings.
- Improve Spark job performance by adjusting partitioning, shuffle, caching, and resource allocation for production-level reliability and efficiency.
- Establish and uphold data engineering standards, patterns, and best practices within the team.
- Design systems focused on long-term maintainability, including clear contracts, testable components, and well-considered failure modes.
- Work with platform and infrastructure teams to advance the foundational architecture of the enterprise data ecosystem.
- Develop and sustain MCP (Model Context Protocol) servers that provide access to data assets and engineering workflows for AI models and internal tools.
- Collaborate with platform teams to incorporate agentic workflows into the data engineering lifecycle.
- Assess and implement emerging AI-native tools for data engineering, staying current on how LLMs and agents can expedite data tasks.
- Promote, exemplify, and integrate cultural principles such as customer success focus, long-term building, growth mindset, inclusivity, and teamwork as the company expands globally.
- Offer mentorship to junior team members and provide technical guidance, training, and knowledge-sharing across departments.
- Interact directly with internal cross-functional stakeholders to comprehend their data requirements and devise scalable solutions.
- Lead comprehensive projects as the primary contact for stakeholders.
Work Arrangement
Remote position open to candidates residing in Canada
