Anthropic is seeking a Senior Software Engineer to join our team dedicated to building and maintaining the critical systems that serve Claude to millions of users globally. You will be responsible for the entire stack, from intelligent request routing to fleet-wide orchestration across diverse AI accelerators. Your work will maximize compute efficiency to serve customer growth while enabling breakthrough research by providing high-performance inference infrastructure.
What You'll Do
- Design intelligent routing algorithms that optimize request distribution across thousands of accelerators.
- Autoscale compute fleet to dynamically match supply with demand across production, research, and experimental workloads.
- Build production-grade deployment pipelines for releasing new models to millions of users.
- Integrate new AI accelerator platforms to maintain hardware-agnostic competitive advantage.
- Contribute to new inference features like structured sampling and prompt caching.
- Support inference for new model architectures.
- Analyze observability data to tune performance based on real-world production workloads.
- Manage multi-region deployments and geographic routing for global customers.
What We're Looking For
- Significant software engineering experience, particularly with distributed systems.
- At least a Bachelor's degree in a related field or equivalent experience.
Nice to Have
- Experience with high-performance, large-scale distributed systems.
- Experience implementing and deploying machine learning systems at scale.
- Background in load balancing, request routing, or traffic management systems.
- Knowledge of LLM inference optimization, batching, and caching strategies.
- Experience with Kubernetes and cloud infrastructure like AWS or GCP.
- Proficiency in Python or Rust.
- A results-oriented approach, with a bias towards flexibility and impact.
- Initiative to pick up slack, even outside of a strict job description.
- Desire to learn more about machine learning systems and infrastructure.
- Ability to thrive where technical excellence directly drives both business results and research breakthroughs.
- Care for the societal impacts of your work.
Technical Stack
- Kubernetes
- AWS, GCP
- Python, Rust
Team & Environment
You will join the Inference team at Anthropic, an extremely collaborative group that works as a single cohesive unit on a few large-scale research efforts. We greatly value communication skills and strive to include a range of diverse perspectives.
Benefits & Compensation
- Competitive compensation and benefits.
- Optional equity donation matching.
- Generous vacation and parental leave.
- Flexible working hours.
- Lovely office space.
- Compensation range: £225,000—£325,000 GBP.
Work Mode
This role is a hybrid position based out of Anthropic offices.
Anthropic is an equal opportunity employer committed to creating a diverse and inclusive team.
