Create high-quality prompts and responses across a wide range of topics. Conduct annotation and evaluation tasks for large language models, including ranking, scoring, labeling, and tagging outputs. Assess model-generated responses for accuracy, relevance, and adherence to instructions. Detect and document issues such as factual inaccuracies, hallucinations, and logical inconsistencies. Participate in and support labeling workflows involving hands-on annotation and coordination with internal or external teams. Train annotation teams on best practices for developing large language models and datasets
Responsibilities
- Creatively generate prompts and responses on diverse subjects
- Perform LLM annotation and evaluation including ranking, scoring, labeling, and tagging
- Evaluate model outputs for accuracy, relevance, and instruction compliance
- Identify and document problems such as hallucinations and inconsistencies
- Engage in labeling workflows with hands-on annotation and collaboration across teams
- Train teams on best practices for large language model and dataset creation
Requirements
- Proven experience in data annotation or evaluation, such as labeling, ranking, scoring, or tagging LLM outputs
- Native or near-native proficiency in U.S. English with strong writing ability
- High level of attention to detail and consistent adherence to guidelines
- Self-motivated with enthusiasm for working on advanced machine learning systems
- Bachelor's degree from an accredited institution or equivalent practical experience
- Must possess valid work authorization in the United States
Nice to Have
- Degree or experience in Linguistics, English Literature, Creative Writing, Journalism, or subject matter expertise in Law, Medical, Math, or Coding
- Familiarity with annotation platforms or structured labeling environments
- Strong understanding of Large Language Models and Reinforcement Learning from Human Feedback (RLHF)
- Experience labeling or tagging prompts, tasks, or frames for use in deep neural networks
- Background in QA or software testing
Benefits
- Full-time W-2 employee with benefits
- Employment eligibility verification through live video and ID submission
Compensation
$27.38/hour
Work Arrangement
Remote (USA), 40 hours per week, 5 days a week
Additional Information
- Start date is April 2026
- Position is full-time W-2 with benefits, requiring five days of work per week
- Applicants must have native or near-native proficiency in U.S. English
- Valid U.S. work authorization is required; no visa sponsorship is offered
- New hires must complete live video identity verification and submit photos of IDs within the first three days of employment
- All employees must verify identity and work eligibility and complete required employment eligibility forms
- Candidates must pass anti-fraud checks to meet program requirements
No visa sponsorship available
