Responsibilities
- Lead the design and implementation of core components of Temporal’s OSS replication stack, from initial design through rollout and long-term operational ownership.
- Design and evolve replication protocols that power: High Availability namespaces, Cross-cluster and cross-region replication, Migration between Temporal clusters (cloud ↔ self-hosted, cloud ↔ cloud)
- Build scalability and reliability capabilities such as: Multi-cell namespaces, Protocols enabling a single namespace to span multiple clusters, Dynamic split/merge strategies based on usage, hot spots, and capacity needs
- Reason deeply about correctness: consistency models, ordering guarantees, idempotency, failure recovery, and safe rollouts of protocol changes.
- Drive cross-team alignment with Cloud Enablement and other CGS teams to ensure OSS foundations support current and future cloud products.
- Author high-quality design docs that clarify invariants, trade-offs, failure modes, and operational playbooks for complex changes.
- Raise engineering standards through reviews, mentorship, and technical leadership—improving correctness testing, fault injection, and incident readiness.
- Participate in on-call/incident response related to replication and core system behavior, helping build durable fixes and prevention mechanisms.
Requirements
- 10+ years building production systems, including significant experience with distributed systems and correctness-critical infrastructure.
- Strong experience with replication, consistency, fault tolerance, and failure recovery in distributed environments.
- Demonstrated ability to design and implement concurrent, correctness-critical systems with clear invariants and safety guarantees.
- Proven track record of leading complex technical projects across teams—setting direction, driving execution, and landing changes safely in production.
- Hands-on experience debugging complex production issues involving race conditions, data consistency, partial failures, and performance degradation.
- Proficiency writing production-quality concurrent code, preferably in Go (Java/C++ or similar systems languages also welcome).
- Solid understanding of distributed systems fundamentals such as replication, sharding/partitioning, backpressure, failure detection, and durability mechanisms.
- Ability to operate with high ownership and minimal oversight, balancing deep technical rigor with pragmatic delivery.
- Curiosity and rigor in understanding how systems behave under stress, failure, and scale.
Nice to Have
- Experience designing or maintaining replication protocols or data-plane infrastructure.
- Experience with multi-cluster or multi-region architectures (active-active / active-passive).
- Familiarity with database internals, log-based replication, or event-sourced systems.
- Prior contributions to large OSS projects or distributed systems infrastructure.
Benefits
- Unlimited PTO, 12 Holidays + 2 Floating Holidays
- 100% Premiums Coverage for Medical, Dental, and Vision
- AD&D, LT & ST Disability, and Life Insurance (Standard & Supplemental Available)
- Empower 401K Plan
- Additional Perks for Learning & Development, Lifestyle Spending, In-Home Office Setup, Professional Memberships, WFH Meals, Internet Stipend and more!
- Paid Time Off (PTO) and Benefits outside the United States vary by country, and are issued in partnership with Remote.com.
- Additionally, Temporal offers perks to all international employees for learning & career development, a lifestyle spending account, in-home office setup (in addition to company-issued hardware), professional memberships, work-from-home meals, and access to the Calm app for mental wellness.
Work Arrangement
Remote (Worldwide)
Team
Structure: part of Temporal’s Cloud Global Services (CGS) organization
Additional Information
- Occasional travel may be required for company events, team offsites, and other meaningful moments that bring us together.
- Temporal is an Equal Opportunity Employer. Temporal Technologies does not discriminate on the basis of race, religion, color, sex, gender identity, sexual orientation, age, non-disqualifying physical or mental disability, national origin, veteran status, or any other basis covered by appropriate law. All employment is decided on the basis of qualifications, merit, and business need. We embrace and celebrate differences and diversity. Temporal is committed to providing access, equal opportunity, and reasonable accommodation for individuals with disabilities in employment, its services, programs, and activities. If you need to request a reasonable accommodation, please let your Recruiter know so we can assist. We are not working with external recruitment agencies, thanks.
