Thai AI Data Annotator, Quality Analyst

Posted 5hrs ago

Employment Information

Education
Salary
Experience
Job Type

Report this job

Job expired or something wrong with this job?

Job Description

Thai AI Data Annotator evaluating Gemini personalization using personal Google data. Ranking model responses, verifying grounding, and writing detailed quality rationales.

Responsibilities:

  • Evaluate a new Gemini personalization feature using information from past Gemini conversations, Gmail, Google Search, and YouTube activity
  • Design and execute multi-turn conversational prompts, typically 1–5 turns, using personal information and experiences
  • Evaluate model responses against the intent of the starting prompt
  • Analyze Grounding issues and verify that claims about the user are supported by evidence
  • Assess Integration quality and whether personal data is incorporated naturally
  • Stack-rank two model responses side-by-side for helpfulness, usability, and enjoyment
  • Write clear, defensible rationales referencing issues or positive aspects in specific conversation turns
  • Extract and verify Debug Info to confirm chat summaries and data sources were properly utilized
  • Maintain data hygiene by deleting evaluation conversations

Requirements:

  • Ability to read and write Thai with a high degree of proficiency
  • Willingness to use a primary personal Google account and enable personal data sources
  • Full-time availability in the local time zone
  • Minimum 30 hours per week and at least 4 hours per day
  • Four hours of overlap with PST
  • Exceptional analytical thinking for evaluating nuanced and ambiguous AI responses
  • Experience designing creative, multi-turn prompts based on personal context
  • Understanding of personalization concepts, including incorrect personalization, poor inferences, and forced connections
  • Ability to review Side-by-Side (SxS) model responses and identify subtle differences in naturalness and overnarrating
  • Excellent written communication and ability to write clear, concise, structured rationales referencing specific turn numbers
  • Ability to provide constructive feedback and detailed annotations
  • Excellent communication and collaboration skills
  • Ability to work independently in a remote setting
  • Desktop/laptop with a good internet connection
  • BS/BA degree or equivalent experience in a relevant field
  • Experience in data annotation, AI quality evaluation, content moderation, or a related role is strongly preferred

Benefits:

  • Full-time availability with flexible scheduling options of 30 or 40 hours per week
  • Contractor engagement
  • Remote work arrangement