Software, AI, IT, Data Evaluator
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
AI evaluator reviewing software, IT, and data artifacts for Mercor’s AI-model training projects. Grading quality, spotting errors, and delivering structured expert feedback.
Responsibilities:
- Review and assess AI-generated work products, including documents, spreadsheets, and slide decks
- Apply deep subject-matter expertise to grade AI-generated outputs
- Evaluate AI-generated artifacts against domain-specific quality rubrics
- Identify factual, aesthetic, and presentation errors
- Provide clear, structured written feedback
- Work on projects focused on training and enhancing AI systems
Requirements:
- 5+ years of relevant professional experience in Software / AI / IT / data
- Native or professional fluency in English
- Highly proficient in Microsoft Office and Google Workspace, especially Google Slides / PowerPoint
- Advanced degree (Master's or higher) from a reputable institution (preferred)
- Must be able to work as an independent contractor
- H1-B and STEM OPT candidates are not supported
Benefits:
- Fully remote role
- Flexible schedule; work can be completed on your own schedule
- Weekly payments via Stripe or Wise based on services rendered
- Competitive pay
- Collaborate with leading researchers
- Referral bonus of up to $480 per successful referral
- Reasonable accommodations upon request
Similar Jobs
AI Safety Expert, English – Finnish
AI red-team expert probing conversational models for Mercor, which partners with AI labs to train frontier systems. Generating vulnerability data, attack cases, and reproducible safety reports.
AI Safety Expert – English, Punjabi
AI red-teaming contractor probing conversational models for jailbreaks, bias, and misuse. Producing reproducible vulnerability data for Mercor’s frontier AI training projects.
AI Safety Expert, English – Punjabi
AI red-teaming contractor probing conversational models for Mercor, which partners with AI labs to train frontier systems. Generating reproducible vulnerability data and attack cases.
Linguistic Projects – Hindi, Tamil, English
Freelance Hindi-Tamil-English annotator supporting Sigma.AI’s global AI data collection and annotation projects. Performing remote transcription, validation, localization, and linguistic evaluation tasks.
Quality Control Specialist, AI/ML, Turkish
Quality control specialist reviewing Turkish AI speech-transcription annotations for TSMG. Correcting formatting, punctuation, capitalization, and normalization issues against project guidelines.
Quality Control Specialist – AI/ML, Thai Language
Quality control specialist reviewing Thai display transcriptions for TSMG’s AI speech-recognition project. Correcting punctuation, formatting, capitalization, and text-normalization issues.
Quality Control Specialist, AI/ML, Russian Language
Quality control specialist reviewing and correcting Russian AI-generated transcripts for TSMG’s speech-recognition project. Ensuring accurate formatting, punctuation, capitalization, and text normalization.
AI Safety Red Teamer
AI Safety Red Teamer stress-testing frontier models for Mercor’s AI training partnerships. Designing adversarial prompts, finding vulnerabilities, and improving model safety with AI researchers.
AI Safety Red Teamer
AI Safety Red Teamer adversarially testing frontier AI models for Mercor’s AI training projects. Identifying jailbreaks, hallucinations, and policy failures across sensitive domains.
AI Safety Red Teamer
AI Safety Red Teamer stress-testing frontier AI models for Mercor’s AI training partnerships. Identifying vulnerabilities and improving model alignment, robustness, and safety.
AI Automation, Integration Specialist
AI automation specialist scaling Primal Storm’s natural-products eCommerce operations. Building Python and API workflows to reduce repetitive work across the company.
Generative AI Analyst
Generative AI Analyst reviewing and annotating Polish content for Welo Data, a global AI data company. Improving datasets and quality processes for multilingual generative AI systems.
AI Safety Expert, English – Dutch
AI safety red teamer probing conversational AI models for Mercor’s human-data AI training projects. Testing vulnerabilities, generating reproducible datasets, and strengthening frontier AI systems.
Marketing Expert – Paid Growth, AI Agent Environments
Paid Growth expert designing and evaluating simulated acquisition tasks for Mercor’s AI training environments. Diagnosing campaigns, budgets, funnels, and journeys using Google Ads, Meta Ads, and GA4 expertise.
AI Safety Expert, English – Dutch
AI safety red team experts probing Mercor-trained frontier AI models for vulnerabilities. Testing conversational agents and producing reproducible datasets, reports, and attack cases.
AI Safety Red Teamer
AI Safety Red Teamer stress-testing frontier AI models for Mercor, which partners with AI labs and enterprises. Identifying vulnerabilities and improving model safety through adversarial evaluation.
AI Safety Expert, English & Assamese
AI red teaming experts probing conversational models for Mercor, which partners with AI labs and enterprises to train frontier systems. Generating vulnerability data and reproducible attack artifacts.
Senior Design Expert – Paid AI Design Research Study
Senior designers shaping AI design tooling through recorded research sessions for Mercor. Evaluating design quality, critique, and shipping standards while helping train AI systems.
AI Safety Expert, English, Finnish
AI red-teaming experts probing frontier models for Mercor’s human-data AI training projects. Generating vulnerability annotations, attack cases, datasets, and reproducible safety reports.
AI Safety Red Teamer
AI Safety Red Teamer identifying vulnerabilities in Mercor’s frontier AI projects. Designing adversarial prompts and evaluating model safety across high-risk domains.