About the Role
We are looking for a senior backend engineer highly skilled Python Developer with strong system design skills and experience building production infrastructure and designing, building, and scaling distributed systems and APIs. In this role, you will own the evolution of the maintain backend platform behind our AI inference endpoints — a multi-tenant system handling authentication, API key management, usage metering, billing and request proxy services that enable high-availability, data-intensive AI-powered solutions. The role is high-ownership and you will shape architecture decisions, raise the bar of engineering quality and build systems that are performant, secure and observable at scale. You’ll collaborate with experts in AI infrastructure and data engineering to build robust, secure, and efficient systems that scale to millions of requests.
Responsibilities
- Design and build core platform services: API gateway, authentication, authorization, key rotation, and multi-tenant isolation.
- Design, implement, and optimize APIs and backend systems using Python frameworks such as primarily FastAPI (or Flask, or Django).
- Architect and implement usage metering, billing integration, and rate limiting for inference endpoints.
- Build and maintain scalable, fault-tolerant microservices for data processing and AI integration.
- Build and operate a high-throughput proxy/routing layer for AI model serving traffic.
- Collaborate with cross-functional teams to design system architecture and ensure seamless system interoperability.
- Design telemetry and observability into the platform from the ground up — structured logging, distributed tracing, metrics, and alerting.
- Implement robust CI/CD pipelines, monitoring, and observability for high-performance production systems.
- Drive technical decisions on architecture, data modeling, and technology choices.
- Identify performance bottlenecks and drive improvements in reliability, scalability, and latency.
- Establish engineering standards for the backend codebase: testing, code review, CI/CD, and deployment practices.
- Ensure best practices for security, compliance, and maintainability across the API lifecycle.
- Collaborate closely with the ML infrastructure team to integrate with model serving systems (e.g., NVIDIA Dynamo, vLLM, SGlang, TensorRT-LLM).
Requirements
- Highly skilled Python Developer with strong system design skills
- Experience building production infrastructure
- Deep experience designing, building, and scaling distributed systems and APIs
