Requirements
- ML Platform expertise: 5+ years building and shipping ML/LLM systems to production
- ML Inference (PyTorch, TensorRT, NVIDIA Triton)
- LLM Inference (LangChain/LangGraph, vLLM, OpenAI/Gemini/Anthropic APIs)
- Compute orchestration (Kubernetes, Prefect, Ray)
- Cloud Infrastructure (AWS, Terraform, VPC, Networking)
- Observability (Prometheus, Grafana, OpenTelemetry, LangSmith/Langfuse)
- Data (ClickHouse, Postgres, Redis)
- Web services (Express/FastAPI, REST, SSE, JWTs)
- Backend JS (e.g. NodeJS) familiarity required
- Familiarity with Agentic Systems: Hands-on experience with LLM agents including: Agent Design: tool use (via MCP), retrieval, memory, grounding/attribution for claims, and guardrails. Architectural patterns: planning and hand-off for multi-agent systems, context management RAG: vector/hybrid search (e.g. pgvector, turbopuffer, chroma), re-rankers (e.g. Cohere, JinaAI)
- Experience with LLM Evaluations at scale: You’ve built offline/online eval harnesses and are familiar with the methodologies and metrics to measure: Search, retrieval, and recommendation performance Agentic task success, trajectory quality, preference learning (SFT, DPO, RLHF, LLM-as-judge) Safety & robustness (security, compliance, red-teaming, regression testing) Cost, performance and latency trade-offs
Nice to Have
- Typescript and Python familiarity welcome
Benefits
- Flexible PTO: We offer non-accrual PTO, plus 11 company holidays.
- Fully-paid health benefits plan for employees: including Medical, Dental, and Vision and an HSA match.
- Family Leave: All employees receive 12 weeks of 100% paid parental leave. Birthing parents are eligible for an additional 6-8 weeks of physical recovery time.
- Fertility & Family Benefits: We have partnered with Maven, a complete digital health benefit for starting and raising a family. Flock will provide a $50,000-lifetime maximum benefit related to eligible adoption, surrogacy, or fertility expenses.
- Spring Health: Spring Health offers a variety of mental health benefits, including therapy, coaching, medication management, and digital tools, all tailored to each individual's needs.
- Caregiver Support: We have partnered with Cariloop to provide our employees with caregiver support
- Carta Tax Advisor: Employees receive 1:1 sessions with Equity Tax Advisors who can address individual grants, model tax scenarios, and answer general questions.
- ERGs: We want all employees to thrive and feel like they belong at Flock. We offer four ERGs today - Women of Flock, Flock Proud, LEOs and Melanin Motion. If you are interested in talking to a representative from one of these, please let your recruiter know.
- WFH Stipend: $150 per month to cover the costs of working from home.
- Productivity Stipend: $300 per year to use on Audible, Calm, Masterclass, Duolingo and so much more.
- Home Office Stipend: A one-time $750 to help you create your dream office.
Additional Information
- If an offer is extended and accepted, this position requires the ability to obtain and maintain Criminal Justice Information Services (CJIS) certification as a condition of employment. Applicants must meet all FBI CJIS Security Policy requirements, including a fingerprint-based background check.
