AI Session Annotator, English
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
AI session annotator evaluating English voice-and-camera assistant responses for Mercor’s AI training projects. Scoring visual accuracy, safety warnings, timing, and spatial-language use with evidence-based comments.
Responsibilities:
- Review recorded sessions involving users speaking to an AI assistant while their camera streams video
- Compare assistant responses with visible video content
- Identify hallucinated or fabricated visual details
- Score sessions for visual interpretation, accuracy, relevance, response timing, and use of spatial/directional language
- Evaluate whether safety warnings concerning health, physical safety, or financial security were timely and proportionate
- Write detailed comments supporting each score with concrete evidence
- Capture conversations turn by turn in the intake form
- Complete a short calibration task if requested
- Work on AI model training and enhancement projects
Requirements:
- Native or native-level English, spoken and written
- Clear, precise written English
- Android phone with a working camera and microphone (mandatory)
- Attention to detail
- Ability to apply a rubric consistently
- Preferred: Prior data annotation, linguistic QA, content moderation, or AI model evaluation experience
- Preferred: Training in linguistics, translation, or interpreting
- Ability to recognize unnatural or non-native English and regional register
- Independent contractor status
- Must not require H1-B or STEM OPT support
Benefits:
- Weekly payments via Stripe or Wise based on services rendered
- Fully remote work
- Flexible schedule / work on your own schedule
- Competitive pay
- Opportunity to collaborate with leading researchers
- Opportunity to help shape the next generation of AI systems
- Referral payments of up to $120 per successful referral
Similar Jobs
Data Privacy and Responsible AI Manager
Data privacy and responsible AI manager for Accuray’s cancer radiotherapy systems. Assessing privacy, vendor, AI, and regulatory risks across enterprise operations.
Sales Leader – Native-AI Practice
Enterprise sales leader driving net-new OpenAI consulting and implementation engagements for Blend, an AI services provider. Building North American pipeline, closing strategic accounts, and developing co-sell motions.
Gen AI Creative Technologist
Gen AI Creative Technologist building agentic workflows and creative systems for Pencil’s AI marketing platform. Enabling faster, scalable, on-brand advertising production.
Marketing Expert – Organic Growth, AI Agent Environments
Organic Growth expert designing and evaluating simulated SEO/AEO marketing tasks for Mercor. Creating realistic environments and rubrics to train and improve frontier AI models.
AI Session Annotator, French
French AI session annotator reviewing voice-and-camera assistant sessions for Mercor, which helps AI labs train frontier models. Scoring visual accuracy, language, timing, and safety warnings.
AI Safety Expert, English – Bengali
AI safety red teamer probing frontier AI models for Mercor. Testing adversarial vulnerabilities and producing human data, datasets, and actionable reports.
Marketing Expert – Lifecycle, AI Agent Environments
Lifecycle marketing expert designing realistic CRM and retention tasks for Mercor’s AI-agent training environments. Evaluating agent decisions, reference answers, and multi-tool workflows.
Generative AI Analyst, Traditional Chinese
Generative AI Analyst evaluating and annotating AI-generated content for Welo Data, a global AI data company. Reviewing multilingual data quality and providing structured feedback.
AI Systems, Zoho & Automation Specialist
AI automation specialist connecting Zoho, AI, CRM, WhatsApp, and data systems. Building reliable workflows for Panama real-estate brokerages.
AI Team Lead
AI Team Lead leading engineers and delivering automation, RPA, and LLM solutions. Driving AI strategy and adoption across Immense Group’s iGaming business.
Marketing Expert – Lifecycle, AI Agent Environments
Lifecycle marketing expert designing simulated environments to evaluate AI agents on real CRM and retention work. Creating briefs, rubrics, and reference answers for Mercor’s AI training projects.
Marketing Expert – Field Marketing, AI Agent Environments
Field marketing expert designing realistic task environments for Mercor’s AI training projects. Evaluating agent performance across events, pipeline recovery, sponsorships, and multi-tool workflows.
Senior Technical Director, AI/ML
AI/ML Senior Technical Director prioritizing cross-business AI initiatives for GE Electrification. Driving strategic roadmaps, risk management, and GenAI implementation with global teams.
Contract Trainer for Operations Management, Data & AI
Operations-Management-Honorartrainer:in für WBS TRAININGs digitale Weiterbildung. Coaching, Projektbewertung und praxisnaher Einsatz von Daten- und KI-Tools im Homeoffice.
Contract Trainer for Operations Management, Data & AI
Honorartrainer:in für Operations Management, Data und KI bei WBS TRAINING. Begleitung von Lernenden im Flipped-Classroom-Modell aus dem Homeoffice.
AI Enablement Engineer
AI Enablement Engineer building internal AI systems, agents, and integrations for Replicant's AI-powered customer-service platform. Enabling teams, managing AI foundations, and improving enterprise AI cost visibility.
Mid-Level Analyst/Developer, Java, Artificial Intelligence
Analista Java diagnosticando e documentando plataforma educacional interna com IA na Deliver IT. Mapeando requisitos, integrações e modernização sem desenvolvimento inicial.
AI Quality Evaluator, German
AI Quality Evaluator assessing German content, language quality, and cultural fit for Canva’s AI design experiences. Building datasets and refining labels with global localisation teams.
AI Safety Expert, English, Portuguese
AI safety experts red teaming conversational models for Mercor’s human-data AI projects. Probing vulnerabilities and generating reproducible datasets to improve frontier AI safety.
Software Evaluator – AI, IT, Data
Evaluator reviewing AI-generated software, IT, and data artifacts for Mercor. Grading quality, identifying errors, and providing structured feedback to train frontier AI systems.