Raising the Village (RTV) seeks a Machine Learning Engineer to build, deploy, and continuously improve our production LLM applications, which are currently live and actively used by field teams and program staff across Uganda, Rwanda, and the Democratic Republic of Congo. You will focus on advancing agentic LLM architectures, RAG systems, and evaluation infrastructure as we scale our AI capabilities to new countries and deepen integration with mobile field tools and our data warehouse.

What You'll Do

Design and implement agentic LLM architectures including multi-step reasoning pipelines, tool use, memory management, and autonomous workflow orchestration using LangChain and related frameworks.
Build, maintain, and optimize Retrieval-Augmented Generation (RAG) pipelines for context-grounded LLM responses, including embedding strategy design and retrieval optimization.
Manage and evolve RTV's vector database infrastructure (Chroma or Qdrant) including index management, namespace organization, and multi-domain retrieval tuning.
Design, build, and maintain end-to-end ML pipelines covering data ingestion, feature engineering, model training, evaluation, and deployment.
Apply knowledge of core ML algorithms to select appropriate modeling approaches for diverse problem types.
Develop and manage the full LLM application lifecycle—from prompt engineering and chain construction through deployment and production monitoring—using LangChain and LangSmith.
Design and implement LLM evaluation frameworks using LLM-as-a-judge approaches, automated metrics, and human evaluation protocols.
Instrument production LLM applications with LangSmith tracing, logging, and feedback collection pipelines for continuous performance monitoring.
Build and deploy RESTful API endpoints for LLM-powered services, ensuring stable integration with mobile field applications.
Develop and maintain personalized content generation pipelines that leverage household segmentation, behavioral data, and program-specific context.
Implement offline and low-connectivity strategies including message caching and fallback mechanisms for use in remote locations.
Collaborate with the Applied Learning team to incorporate validated program content into knowledge bases and generation templates.
Write clear technical documentation for agent architectures, RAG pipeline designs, evaluation frameworks, and API specifications.

What We're Looking For

Bachelor's or Master's degree in Computer Science, Artificial Intelligence, Data Science, Statistics (Computing Major) or a related quantitative field.
3+ years of hands-on experience building and deploying production LLM applications, with a demonstrable portfolio.
Proficiency in LangChain for agentic pipeline construction, tool use, memory integration, and RAG implementation.
Proficiency in LangSmith for LLM application tracing, evaluation, dataset management, and production monitoring.
Proficiency in vector databases (Chroma and/or Qdrant) including embedding management, indexing, and retrieval optimization.
Proficiency in agentic design patterns including ReAct, plan-and-execute, multi-agent orchestration, and tool-augmented reasoning.
Proficiency in LLM evaluation methodologies including LLM-as-a-judge frameworks, reference-based and reference-free metrics, and human-in-the-loop workflows.
Proficiency in Python for LLM application development, API construction (FastAPI or equivalent), and pipeline automation.
Proficiency in OpenAI API and prompt engineering best practices including few-shot prompting, structured output generation, and system prompt design.
Proficiency in cloud deployment on AWS, including containerized application hosting, environment management, and API infrastructure.
Experience integrating LLM applications with structured data sources (SQL databases, data warehouses) for analytics-augmented generative AI capabilities.
Solid understanding of core ML algorithms including supervised and unsupervised learning, classification, regression, ensemble methods, and neural network architectures.
Hands-on experience building and managing ML pipelines including data preprocessing, feature engineering, model training, evaluation, experiment tracking (Weights & Biases or equivalent), and production deployment using CI/CD practices.

Nice to Have

Familiarity with mobile application integration and offline-first design patterns for low-connectivity deployment environments.

Technical Stack

LangChain, LangSmith, Chroma, Qdrant, Python, FastAPI, OpenAI API, AWS, SQL databases, Data warehouses, Weights & Biases

Team & Environment

This role sits within the Predictive Analytics / VENN department at Raising the Village, reporting to a Senior Data Scientist. The organization comprises a team of over 250 in Uganda, plus 17 in North America and 15 in Rwanda.