Software Evaluator – AI, IT, Data
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Evaluator reviewing AI-generated software, IT, and data artifacts for Mercor. Grading quality, identifying errors, and providing structured feedback to train frontier AI systems.
Responsibilities:
- Review and assess AI-generated work products, including documents, spreadsheets, and slide decks
- Apply deep subject-matter expertise to grade AI-generated outputs
- Evaluate AI-generated artifacts against domain-specific quality rubrics
- Identify factual, aesthetic, and presentation errors
- Provide clear, structured written feedback
- Work on projects training and enhancing AI systems
Requirements:
- 5+ years of relevant professional experience in Software / AI / IT / data
- Native or professional fluency in English
- Highly proficient in Microsoft Office and Google Workspace
- Especially proficient in Google Slides / PowerPoint
- Advanced degree (Master's or higher) from a reputable institution (preferred)
- Independent contractor engagement
- H1-B and STEM OPT candidates are not supported
Benefits:
- Fully remote role
- Flexible schedule / work on your own schedule
- Weekly payments via Stripe or Wise based on services rendered
- Competitive pay
- Collaboration with leading researchers
- Opportunity to help shape the next generation of AI systems
- Referral program: earn up to $480 for each successful referral (restrictions may apply)
Similar Jobs
AI Safety Expert, English & Danish
AI safety red teamer probing conversational AI models for Mercor, which partners with AI labs to train frontier systems. Generating reproducible vulnerability data and attack cases.
Senior Public Policy, Political Analysis Expert – AI Evaluation
Senior public policy expert creating regulatory and political-analysis tasks for Gramian Consultancy’s AI evaluation work. Reviewing AI outputs and translating complex evidence into actionable recommendations.
Working Student – AI Sales
Werkstudent im B2B-Vertrieb für autonome KI-Agenten und Sprachautomatisierung bei Lane One. Lead-Generierung, Cold Calling und Terminvereinbarung für das Sales-Team.
AI Automation Specialist – Part Time
AI automation specialist building agents and workflows for project management and commercial real estate. Integrating monday.com, SaaS tools, and AI solutions for teams across LATAM.
Master’s Scholarship Fellow – Applied AI and Automation Researcher
Pesquisador aplicado em IA e automação para a ICT Itaú, instituição brasileira de conexão entre indústria e conhecimento. Desenvolvendo automação, LLMs, artigos e patentes.
AI Systems Integration Specialist
AI integration specialist connecting Airtable, Google Workspace, Notion, and Claude. Optimizing automated workflows and data systems for Hireframe’s remote staffing operations.
Especialista en Inteligencia Artificial
Especialista en IA desarrollando soluciones generativas, LLMs y agentes inteligentes para clientes empresariales de Towa. Evaluación de tecnologías, arquitecturas, automatización e integración en proyectos de innovación.
Regional AI Business Lead
Regional AI sales leader driving complex enterprise AI infrastructure campaigns for Everpure, a data-storage company. Advising executive buyers, partners, and sales teams on scalable AI platforms and ROI.
Subject Matter Expert – AI Team Management
AI team-management course SME shaping Ziplines Education’s certificate content for working professionals. Recording practical lessons on AI adoption, supervision, team norms, and rollout planning.
Principal Enterprise AI and Transformation Architect
Principal architect delivering enterprise AI, automation, and systems integration for Simulations Plus, a drug-development software company. Building production solutions and guiding transformation across regulated enterprise operations.
Principal Technical Consultant – AI
Principal AI consultant delivering production GenAI and agentic workflows for AHEAD’s digital transformation platforms. Building, deploying, and operating reliable AI solutions for enterprise client engagements.
Senior Product Architect, K8s-Based AI Infrastructure
Product Architect designing Kubernetes-based AI infrastructure for NVIDIA’s accelerated computing platform. Shaping scalable architectures, prototypes, and AI solutions from concept through lifecycle.
AI Claims Reviewer – Dutch Language
Dutch AI Claims Reviewer fact-checking AI-generated claims for TrustScale, an AI trust and verification platform. Comparing claims with reliable sources and producing evidence-based verdicts.
Consultant – AI-Powered Insight, Quant / Advanced Analytics
Quantitative analytics consultant at Verve, helping global brands understand people, markets and opportunities. Applying research, advanced analytics and AI to solve complex client challenges.
Visiting Professor – Artificial Intelligence, Online
Visiting Professor teaching artificial intelligence at DeVry University. Developing course materials, facilitating learning, and integrating responsible AI into online instruction.
Principal, AI Solutions
Principal AI solutions leader designing AI agents, workflows, and automations for Higher Logic’s community solutions. Partnering with business teams to improve workflows, adoption, and measurable organizational outcomes.
AI & No-Code Mentor, Vibe Coding Bootcamp
AI- und No-Code-Mentor für Digitale Leute School, die Bootcamps für Product, Design und Tech anbietet. Begleitung von Quereinsteigern beim Bau digitaler Produkte mit AI- und No-Code-Tools.
AI Safety Practitioner
AI safety practitioner evaluating frontier models for Mercor, which partners with AI labs to train and enhance AI systems. Applying safety policies, identifying failures, and providing structured feedback on sensitive content.
AI Safety Practitioner
AI Safety Practitioner evaluating frontier AI models for safety, quality, and alignment. Mercor partners with AI labs and enterprises to train and enhance AI systems.
AI Safety Practitioner
AI Safety Practitioner evaluating frontier AI models for Mercor, which partners with AI labs and enterprises to train advanced systems. Assessing sensitive content, applying safety rubrics, and providing structured alignment feedback.