Thai AI Data Annotator, Quality Analyst
Posted 5hrs ago
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Thai AI Data Annotator evaluating Gemini personalization using personal Google data. Ranking model responses, verifying grounding, and writing detailed quality rationales.
Responsibilities:
- Evaluate a new Gemini personalization feature using information from past Gemini conversations, Gmail, Google Search, and YouTube activity
- Design and execute multi-turn conversational prompts, typically 1–5 turns, using personal information and experiences
- Evaluate model responses against the intent of the starting prompt
- Analyze Grounding issues and verify that claims about the user are supported by evidence
- Assess Integration quality and whether personal data is incorporated naturally
- Stack-rank two model responses side-by-side for helpfulness, usability, and enjoyment
- Write clear, defensible rationales referencing issues or positive aspects in specific conversation turns
- Extract and verify Debug Info to confirm chat summaries and data sources were properly utilized
- Maintain data hygiene by deleting evaluation conversations
Requirements:
- Ability to read and write Thai with a high degree of proficiency
- Willingness to use a primary personal Google account and enable personal data sources
- Full-time availability in the local time zone
- Minimum 30 hours per week and at least 4 hours per day
- Four hours of overlap with PST
- Exceptional analytical thinking for evaluating nuanced and ambiguous AI responses
- Experience designing creative, multi-turn prompts based on personal context
- Understanding of personalization concepts, including incorrect personalization, poor inferences, and forced connections
- Ability to review Side-by-Side (SxS) model responses and identify subtle differences in naturalness and overnarrating
- Excellent written communication and ability to write clear, concise, structured rationales referencing specific turn numbers
- Ability to provide constructive feedback and detailed annotations
- Excellent communication and collaboration skills
- Ability to work independently in a remote setting
- Desktop/laptop with a good internet connection
- BS/BA degree or equivalent experience in a relevant field
- Experience in data annotation, AI quality evaluation, content moderation, or a related role is strongly preferred
Benefits:
- Full-time availability with flexible scheduling options of 30 or 40 hours per week
- Contractor engagement
- Remote work arrangement











