MediaRadar is looking for a Senior Data Engineer to architect, build, and scale robust data pipelines that ensure flawless data quality. You will tackle complex challenges with our modern cloud data stack, delivering timely and reliable data.
What You'll Do
- Design, implement, and optimize scalable and reliable ELT/ETL pipelines using Databricks, Spark, Python, and Store Procedures.
- Model complex data and build solutions for both batch and streaming workloads.
- Guide and support a team of data engineers, providing technical direction and resolving challenges.
- Elevate the team's skills through mentorship and constructive code reviews.
- Develop and implement comprehensive testing frameworks, data validation rules, and QA plans.
- Proactively monitor system performance, tune complex SQL queries, and troubleshoot production issues.
- Perform root-cause analysis and implement lasting solutions to improve system health.
- Actively participate in an Agile environment (Scrum, sprints, backlog grooming) and collaborate with cross-functional teams.
- Help evaluate and introduce new technologies and best practices to continuously improve our data platform.
What We're Looking For
- A bachelor’s or master’s degree in computer science, Engineering, or a related field (or equivalent practical experience).
- 7+ years of hands-on experience in data engineering.
- Strong focus on Databricks, Apache Spark and 1 RDBMS solution for processing large-scale datasets.
- Expert-level proficiency in Python (including PySpark).
- Strong understanding of data structures, algorithms, and OOP principles.
- Deep expertise in SQL and Spark SQL, with proven experience writing and optimizing complex analytical queries.
- Hands-on experience with the Databricks ecosystem, including Delta Lake, Unity Catalog, and Data frame APIs.
Nice to Have
- Experience building and maintaining CI/CD pipelines using tools like Azure DevOps.
- Previous experience migrating a legacy data system to a modern, unified data platform.
- Familiarity with containerization (Docker, Kubernetes).
- Experience working in a fast-paced, product-driven environment.
Technical Stack
- Azure
- Databricks
- Apache Spark
- Python
- SQL
- Delta Lake
- Unity Catalog
Benefits & Compensation
- Salary: $150,000 - $165,000
- Medical, Dental & Vision Insurance
- 401k with Company Match
- Flexible PTO
- Commuter Benefits
- Gym Discounts
- Summer Fridays
Work Mode
This is a remote position.
We are proud to be an Equal Opportunity Employer and make employment decisions without regard to race, color, religion, sex (including pregnancy, sexual orientation, or gender identity), national origin, age, disability, genetic information, or any other legally protected status.
