About the Role
This role involves reviewing and assessing AI-generated text to identify potential safety risks, cultural insensitivity, or policy violations specific to the Malay-speaking audience in Singapore. Evaluators provide feedback that helps improve model behavior and reduce harmful outputs.
Responsibilities
- Review AI-generated responses in Malay for accuracy and tone
- Identify content that may be offensive or inappropriate in the Singaporean context
- Flag responses that violate safety policies or ethical guidelines
- Assess alignment of outputs with local cultural norms
- Provide detailed feedback on problematic content
- Rate the quality and safety of model-generated text
- Follow evaluation guidelines consistently across tasks
- Report edge cases that challenge current safety frameworks
- Contribute to refining safety benchmarks for AI models
- Work with nuanced language use including slang and code-switching
- Evaluate responses across diverse topics such as health, politics, and religion
- Ensure outputs avoid promoting misinformation
- Detect subtle forms of bias or stereotyping
- Maintain high attention to detail during repetitive tasks
- Adapt to evolving evaluation criteria
- Collaborate with team leads to clarify ambiguous cases
- Meet performance targets for accuracy and throughput
- Handle sensitive content with professionalism
- Stay updated on local social and cultural developments
- Support the development of safer AI systems through consistent feedback
Nice to Have
- Formal education in linguistics, social sciences, or related fields
- Experience evaluating AI-generated content
- Background in ethics or philosophy
- Knowledge of Singaporean media and public discourse
- Familiarity with AI safety principles
- Experience with annotation tools or data labeling
- Understanding of hate speech and misinformation patterns
- Prior work in multilingual environments
- Training in cultural studies or anthropology
- Demonstrated interest in AI ethics
Compensation
Competitive hourly rate based on experience and location
Work Arrangement
Remote
Team
Part of a global team focused on AI safety and content evaluation
Project Focus
This project specifically targets AI safety in Malay-language interactions within the Singapore context, emphasizing cultural appropriateness and harm reduction.
Language Requirements
Fluency in Malay is required, with an understanding of local expressions, idioms, and common usage patterns in Singapore.
Evaluation Criteria
Evaluators will use structured guidelines to judge safety, factual consistency, and cultural alignment of AI responses.
Time Commitment
Flexible hours with expected weekly availability; tasks are distributed asynchronously.
Training and Onboarding
Comprehensive training will be provided to ensure evaluators understand safety policies and evaluation protocols.
Performance Metrics
Evaluators are assessed on accuracy, consistency, and adherence to guidelines.
Data Privacy
All content reviewed is confidential; strict data handling protocols must be followed.
Feedback Impact
Evaluator input directly informs improvements to AI model safety and behavior.
Technical Setup
A computer with stable internet access is required to complete evaluation tasks.
Project Duration
Initial assignments may be short-term with potential for extension based on performance and project needs.
Not applicable for remote roles
