Constructor is looking for a Data Analyst to join our 6-person Ranking Team. This team builds and optimizes the machine learning models that determine what products shoppers discover through search. You will evaluate and improve the performance of ranking models, features, and configurations across our large-scale experimentation program.
What You’ll Do
- Analyze A/B tests across models, configurations, and customer segments to quantify business impact and guide ML development.
- Investigate training data quality, model performance trends, and feature effectiveness at scale.
- Investigate how ranking changes affect user behavior and conversion metrics.
- Use SQL, Python, and Spark to uncover usage patterns, anomalies, and opportunities for optimization.
- Define new metrics to measure search relevance, personalization, and model performance.
- Ensure metrics align with user experience and business goals through rigorous validation.
- Create scalable dashboards and reporting tools for product, engineering, and leadership teams.
- Develop debugging tools to explain ranking decisions and identify performance issues.
- Partner cross-functionally to design experiments, validate hypotheses, and communicate insights that influence product roadmap and ML strategy.
What We’re Looking For
- 3+ years analyzing complex experiments and extracting actionable insights from large, noisy datasets.
- Experience with statistical testing and practical experiment design.
- Ability to write optimized SQL queries for terabyte-scale data extraction and transformation.
- Proficiency with distributed systems like Spark for large-scale data processing.
- Strong Python skills in exploratory analysis, custom metrics development, and building internal tools.
- Experience with data science libraries and automation.
- Understanding of ML pipelines, training data quality, and ranking/recommendation metrics.
- Familiarity with search relevance and personalization concepts.
- Ability to design metrics that accurately reflect model and product performance.
- Skill in ensuring alignment between technical metrics and business outcomes.
- Experience creating compelling dashboards using Tableau, Looker, or custom dashboards in Python.
- Ability to present complex findings clearly to both technical and executive audiences.
- Skill in influencing product and engineering decisions through data storytelling.
- Ability to collaborate effectively across teams to drive ML and product improvements.
- Deep curiosity about user behavior and business impact.
- Ability to connect algorithm changes to real-world customer outcomes.
Technical Stack
- SQL
- Python
- Spark
- Tableau
- Looker
Team & Environment
You will be the first data analyst on this 6-person ranking team, collaborating closely with engineers, data scientists, and product managers.
Benefits & Compensation
- Compensation: $80k-$120k USD
- Unlimited vacation time with encouragement to take at least 3 weeks per year.
- Fully remote team — choose where you live.
- Work from home stipend.
- Apple laptops provided for new employees.
- Training and development budget for every employee, refreshed each year.
- Maternity & Paternity leave for qualified employees.
- Regular team offsites to connect and collaborate.
Work Mode
This is a fully remote, global position. You can choose where you live and work.
At Constructor we are committed to cultivating a work environment that is diverse, equitable, and inclusive. As an equal opportunity employer, we welcome individuals of all backgrounds and provide equal opportunities to all applicants.
