Responsibilities
- Design and architect AI agents
- Design reusable AI agent patterns for Product workflows such as research synthesis, PRD (Product Requirement Document) drafting support, roadmap analysis, operational triage, and knowledge retrieval.
- Define agent instructions, tool schemas, guardrails, and structured output contracts.
- Architect human-in-the-loop decision boundaries where appropriate.
- Integrate agents into Microsoft 365, Copilot Studio, Power Platform, AWS services, and internal systems.
- Define and maintain action interfaces between agents and deterministic systems (APIs, workflows, runbooks).
- Ensure reliable orchestration across conversational and workflow layers.
- Own quality, evaluation, and governance
- Define measurable success criteria for agent outputs.
- Build lightweight evaluation frameworks and regression testing for prompts and behavior.
- Monitor output quality, hallucination risk, cost, and drift over time.
- Establish safe usage guidelines and governance boundaries.
- Operate and improve in production
- Monitor agent usage, reliability, and impact.
- Optimize performance, token efficiency, and response structure.
- Iterate on agent behavior based on real usage patterns and feedback.
- Proactively identify new opportunities where AI agents can improve Product workflows.
Requirements
- Strong ownership
- Product empathy
- Comfort operating independently in a production environment with real business impact
- Applied AI engineering skills
- Systems thinking
- Experience in production-grade agent design, evaluation, governance, and workflow integration
