At Truelogic Software, we're hiring a Senior Quality Engineer (AI) to join our mission in e-learning. This strategic and hands-on role blends test innovation, prompt effectiveness analysis, and user feedback insights for Generative AI systems. You will help shape and evolve QA practices specifically for GenAI, partnering closely with developers, product owners, and SMEs to ensure robust, safe, and high-quality AI features.

What You'll Do

Develop a QA framework tailored to GenAI systems and workflows.
Design tests for prompt behavior across varied inputs, hallucination detection, and factual consistency.
Blend manual and automated test design for both deterministic and stochastic outputs.
Collaborate with teams to obtain or create sample data with clear target outputs.
Own end-to-end test strategy and execution for GenAI-powered features.
Ensure coverage across diverse prompt phrasing, user intents, failure modes, multiple GenAI features, and high-risk scenarios.
Lead the design and implementation of prompt and model evaluation protocols for alignment, fluency, tone, coherence, clarity, coverage, and relevance.
Use Golden datasets and benchmark prompts to establish evaluation baselines.
Design and manage SME-driven Human-in-the-Loop (HITL) review workflows.
Facilitate structured reviews focused on correctness/accuracy and capturing edge-case failures.
Define and track QA effectiveness using metrics like pass rate for high-risk use-cases and HITL reviewer agreement rates.
Deliver clear, actionable dashboards and reports to leadership on AI quality, safety, and readiness.

What We're Looking For

You are excited by the complexities and challenges of GenAI testing.
You think like a product owner, act like a tester, and communicate like a coach.
You thrive in ambiguity and enjoy shaping new standards.
You are passionate about safe, responsible AI development.