Software Engineer, AI, IT, Data Evaluator
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
AI evaluator reviewing software, IT, and data artifacts for accuracy and quality. Providing structured feedback to help Mercor and leading AI labs train frontier models.
Responsibilities:
- Review and assess AI-generated work products, including documents, spreadsheets, and slide decks
- Apply deep subject-matter expertise to grade outputs
- Evaluate AI-generated artifacts against domain-specific quality rubrics
- Identify factual, aesthetic, and presentation errors
- Provide clear, structured written feedback
- Work on projects focused on training and enhancing AI systems
Requirements:
- 5+ years of relevant professional experience in Software / AI / IT / data
- Native or professional fluency in English
- Highly proficient in Microsoft Office and Google Workspace, especially Google Slides / PowerPoint
- Must be able to work as an independent contractor
- H1-B or STEM OPT candidates are not supported
- Advanced degree (Master's or higher) from a reputable institution preferred
Benefits:
- Fully remote role
- Flexible schedule; work can be completed on your own schedule
- Weekly payments via Stripe or Wise based on services rendered
- Competitive pay
- Opportunity to collaborate with leading researchers
- Opportunity to help shape next-generation AI systems
- Referral payments of up to $480 for each successful referral
- Reasonable accommodations upon request
Similar Jobs
AI Engineer
AI Engineering intern improving prompts, evaluations, and Python validation for PiggyBank Ventures’ Nestora AI mobile app. Shipping reliable AI features within iOS and mobile constraints.
Junior Applied AI Engineer
Junior Applied AI Engineer supporting applied AI design, development, and deployment for an AWS and Databricks partner. Building engineering skills through production delivery, code reviews, and knowledge sharing.
Senior Java Developer – AI-Native
Senior Java Developer designing AI-native cloud-edge software, automation, and optimization applications. Delivering scalable network solutions for tier-1 cellular operators through a wireless technology innovator’s Edgewise platform.
Senior Full Stack Engineer – AI
Senior Full Stack Engineer building AntragPlus, an AI-powered platform for grant-funding infrastructure. Owning full-stack engineering, production AI pipelines, infrastructure, and technical architecture.
Junior Full Stack Engineer – AI
Junior Full Stack Engineer building AI-powered software for grant funding. Developing full-stack features, AI pipelines, and production systems with CTO mentorship.
Senior AI Engineer/Developer
AI Engineer developing CloudPSO’s enterprise voice-agent platform for support, sales, and operations. Building speech, NLP, MLOps, APIs, and frontend prototypes.
Technical Director – AI Engineering, Technology and Digital
Technical Director leading AI engineering, architecture, and high-performance teams for Baptist Health’s South Florida healthcare system. Driving scalable technology solutions across products and cloud platforms.
Senior AI Engineer – Tieto Tech Consulting
Senior AI Engineer building production AI agents, workflows, and Azure platforms for Tieto’s digital consulting clients. Supporting customer engagements, AI operations, and technical team growth.
AI Engineer – Fullstack
AI Engineer building fullstack and generative AI features for CodiLime’s enterprise SaaS platform. Integrating LLMs, APIs, and secure multi-tenant data workflows.
Principal AI Engineer
Principal AI Engineer developing and deploying GenAI and machine-learning solutions. Improving healthcare platforms, automation, and outcomes at Gainwell Technologies.
AI Engineer
AI Engineer building secure Microsoft Copilot, Claude, and private RAG applications. Integrating enterprise data, APIs, governance, and automated business processes.
Principal AI Engineer
Principal AI Engineer leading production AI architecture, technical quality, and engineering capability at Blend data consultancy. Designing scalable AI systems and guiding major client engagements.
Principal AI Engineer – Aerospace & Defense
Principal AI Engineer building AI-powered mobile maintenance software for IFS, whose enterprise platform serves airlines, MROs, OEMs, and defense operators. Deploying and operating solutions directly in customer environments.
Junior Data Product & AI Engineer
Junior Data Engineer bei Europace, Deutschlands FinTech-Marktführer für Finanzierungstechnologie. Integration von Datenquellen und Migrationen auf eine AWS-basierte Data Platform.
AI Engineer
Applied AI Engineer building and operating production AI systems for Inviso’s enterprise clients. Shipping reliable, measurable, safe, and cost-aware AI capabilities.
AI Engineer
Applied AI Engineer building reliable, measurable production AI systems for Inviso’s enterprise clients. Operating LLM, RAG, agent, evaluation, and monitoring capabilities.
AI Engineer
Applied AI Engineer building reliable, evaluated, and monitored production AI systems for Inviso’s enterprise clients. Remote consulting across LLMs, RAG, agents, guardrails, and platform operations.
AI Engineer
Applied AI Engineer building reliable, measurable production AI systems for Inviso’s enterprise clients. Operating LLM, RAG, agent, evaluation, and monitoring capabilities in a remote consulting role.
AI Engineer
Applied AI Engineer building reliable, evaluated, and monitored production AI systems for Inviso’s enterprise clients. Designing agent runtimes, RAG, guardrails, and model-routing capabilities.
Domain AI Architect – DevEx
Domain AI Architect improving developer experience, governance, and Azure platform architecture for a global IT services provider. Driving reusable accelerators, deployment pipelines, and engineering enablement.