Responsibilities
- Designing complex, dynamic prompt templates with conditional logic and efficiently reusing information and context within prompts to maximize generation quality and reasoning.
- Implementing various response schemes (JSON mode, function calling, Zod/JSON schemas) to ensure AI outputs are predictable and ready for seamless integration into application logic.
- Building robust evaluation pipelines and using Langfuse to collect feedback and score the quality of responses in real time.
- Performing deep debugging of complex LLM chains using Langfuse traces to identify bottlenecks and optimize for cost, latency, and context window usage.
- Running systematic experiments across different models via OpenRouter (e.g., comparing Claude 3.5 Sonnet vs. GPT-4o) and analyzing results based on quantitative metrics.
- Making deployment decisions for new prompts or models strictly based on quantitative benchmarks and trace data, rather than intuition.
- Developing scoring systems to analyze the “Problem → Solution” chain and identify root causes of hallucinations or logic errors using Langfuse analytics.
- Regularly re-evaluating model performance as new architectures emerge and performing fine-tuning when necessary to meet specific domain requirements.
Benefits
- Remote Work Environment: Embrace the freedom to work from anywhere, anytime, promoting a healthy work-life balance.
- Unlimited PTO: Enjoy unlimited paid time off to recharge and prioritize your well-being, without counting days.
- Paid National Holidays: Celebrate and relax on national holidays with paid time off to unwind and recharge.
- Company-provided MacBook: Experience seamless productivity with top-notch Apple MacBooks provided to all employees who need them.
- Flexible Independent Contractor Agreement: Unlock the benefits of flexibility, autonomy, and entrepreneurial opportunities. Benefit from tax advantages, networking opportunities, reduced employment obligations, and the freedom to work from anywhere. Read more about it here: https://docs.google.com/document/d/1tzxGX4Uu7Ts_HCLFXESKLnKaaBfVCPf1f9AYZPrkjJM/preview?tab=t.0
Additional Information
- Applicants must be located within approximately ± 4 hours of CET to ensure optimal collaboration and communication during working hours.
- Company-provided MacBook is provided to all employees who need them.
