Responsibilities
- Play a key role in delivering the product roadmap by building features related to data pipelines, usage of LLMs, and analytics APIs
- Perform full stack work as needed to deliver value to customers
- Conduct light React development
- Create notifications, alert emails, scheduled reports, and connectors into third party systems
Requirements
- Extensive experience building & using web APIs, including handling authentication, authorization, caching, rate limiting, etc.
- Experience with relational (transactional) databases, particularly PostgreSQL
- Experience with analytics tools and technologies such as: Analytical SQL, maybe with a modern analytical data platform like Snowflake, Google BigQuery, DuckDB, Spark SQL, etc.
- Experience writing batch and streaming/interactive data processing pipelines
- The ability to quickly understand source data and validate output data quality
- Experience using LLMs and/or traditional ML techniques to build classifiers, data extractors, summarizers, etc.
Nice to Have
- Experience building API for external customers
- Experience with analytics packages like Pandas or Polars
- Experience with streaming analytics
- Experience with big data tools like Flink, Beam (GCP Dataflow), Spark, etc.
- Experience with job queues and task processing frameworks (Celery, Resque, or cloud-based options like Google Cloud Tasks)
- Experience with multi-step durable execution frameworks like Temporal or Inngest
- Experience with in-house or home-rolled versions of data processing systems used at previous companies
- Not required but a big plus: experience building evals for LLM-based tools to assess and improve accuracy at these kinds of tasks
Additional Information
- This role is open to candidates located in and legally authorized to work in the US
- Scrunch is an equal opportunity employer and does not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status
