Responsibilities
- Lead the design and execution of automated testing strategies spanning unit, integration, contract, end-to-end, and edge deployment validation.
- Develop and maintain comprehensive end-to-end test suites using Playwright for user interfaces and human-involved approval processes.
- Create API, contract, and integration tests for Python-based microservices and Kafka event streams, including failure simulation and message replay scenarios.
- Enforce code quality by integrating test suites into CI/CD pipelines with strict pass/fail criteria instead of optional feedback.
- Track test suite performance metrics such as execution duration, flakiness rates, and coverage of critical paths; eliminate root causes of instability.
- Design evaluation frameworks for AI agent behaviors, verifying correct tool usage, parameter passing, escalation logic, and operational limits.
- Implement regression testing approaches for probabilistic components using reference datasets, acceptable variance thresholds, and drift detection after model or prompt updates.
- Automate testing for edge-deployed systems, including offline operation, reconnection logic, and security validation under malicious conditions.
- Collaborate with Technical Business Analysts to transform SOPs and incomplete requirements into automated acceptance tests, and guide developers in test ownership.
Other
- This is a full-time vendor position with a direct contract through Inallmedia.com.
- Candidates must be located within Central Time zone plus or minus two hours.
- Availability to overlap with North American business hours is required.
- All interviews, documentation, and daily communication must be conducted in English.
