Study Domain Expert – Judge Calibration
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Sales expert evaluating and ranking AI-generated outputs for Mercor, which partners with AI labs and enterprises to train frontier models. Providing evidence-based rationales and study feedback.
Responsibilities:
- Apply professional judgment to realistic Sales work and evaluate model-generated outputs
- Sign an NDA before receiving study materials
- Complete one realistic Sales task using supplied source files
- Review model outputs, rank and score them, and write evidence-based rationales
- Provide structured feedback on the task and study design
- Complete 13 hours of work during September 12–15, 2026
- Work on projects focused on training and enhancing AI systems
Requirements:
- More than 5 years of experience in Sales or a closely relevant field
- Experience at a high-caliber firm ideally
- Ideally held a manager, director, or principal-level Sales role; entry-level experience alone is insufficient
- Fluent or native English
- Prior work as an expert is a plus
- Must sign an NDA before receiving study materials
- No use of LLMs or other AI assistants during the study
- People who have worked on Project Alchemy in any capacity or accessed its source-world data are not eligible
- Must be engaged as an independent contractor
- H1-B and STEM OPT candidates are not supported
- No geographic restrictions
Benefits:
- Fully remote role
- Flexible schedule; can be completed on your own schedule
- Weekly payments via Stripe or Wise based on services rendered
- Competitive pay
- Opportunity to collaborate with leading researchers
- Opportunity to help shape the next generation of AI systems
- Referral payments of up to $360 for each successful referral
- Reasonable accommodations available upon request
Similar Jobs
Sales Representative
Sales Representative growing Videojet’s industrial product identification business across Greater San Antonio and Austin. Building customer relationships, prospecting new accounts, and driving regional profitability.
IT Sales Executive
IT Sales Executive selling security, cloud, networking, and software solutions for Sentinel Technologies. Prospecting Charlotte-area clients and closing business through consultative sales.
Technical Sales Specialist – Hydraulics
Technical Sales Specialist growing hydraulic and filtration systems sales for RG Group. Advising customers, managing vendors, and developing business across five Mid-Atlantic states.
Aesthetic Sales Representative, B2B Skincare – Med Aesthetic Experience Required
Aesthetic sales representative growing Tennessee/Kentucky accounts for a professional skincare brand sold through licensed practitioners. Building relationships, educating customers, and driving territory sales.
Sales Executive – Utilization Management
Sales Executive selling Penguin Ai’s AI healthcare workflow platform to payer and provider Utilization Management teams. Owning complex enterprise sales cycles and developing regional go-to-market strategy.
Senior Sales Manager
Sr Sales Manager driving complex B2B sales for First Advantage, a background-screening and workforce-risk solutions company. Building pipelines, winning contracts, and expanding markets.
Asesor de Ventas – Con o Sin Experiencia
Teleoperador remoto vendiendo portabilidad Movistar a clientes de otros operadores. Trabajo comercial para Netcall Perú Group, especialistas peruanos en telecomunicaciones y ventas.
Study Domain Expert – Judge Calibration – Sales
Sales expert completing realistic tasks and evaluating AI-generated outputs for Mercor. Providing ranked scores and evidence-based feedback to improve frontier AI models.
Sales Executive
Sales Executive acquiring restaurant and hospitality customers for Restaurant Technologies’ bulk cooking-oil management services. Managing pipelines, consultative sales, closes, installations, and customer relationships across Greater Portland.
Sales Manager – Africa Region
Sales Manager driving B2B iGaming sales and partnerships across African operator markets. Managing pipelines, complex sales cycles, proposals, and strategic client relationships.
Regional Sales Manager – Work Truck Division
Regional Sales Manager expanding Federal Signal’s Work Truck safety and security product sales across the Northeast. Developing accounts, forecasting pipelines, and supporting distributors.
Regional Sales Manager – Residential
Regional Sales Manager leading residential appliance sales, dealers, and distribution channels across the Southeast. Developing regional strategies, managing sales representatives, and expanding market share for HCC.
Sales Director – Fuels & Lubricants
Sales Director closing enterprise and mid-market deals for Anova’s IIoT tank-monitoring platform. Driving ROI-based sales, renewals, and growth among fuels and lubricant distributors in Germany.
Sales Director – Chemical
Sales Director driving enterprise and mid-market sales of Anova’s IIoT tank-monitoring platform to chemical distributors in Germany. Managing full-cycle deals, executive ROI presentations, forecasting, contracts, and renewals.
Technical Sales Executive
Technical Sales Executive selling EDITED’s AI-driven retail intelligence platform to US enterprise accounts. Explaining APIs and MCP integrations while managing full-cycle enterprise sales.
Area Sales Manager – Southeast Region
Remote mortgage sales leader expanding Capital Bank Home Loans, a nationwide mortgage lender. Recruiting branch managers and originators while driving regional production, profitability, and market growth.
Area Sales Manager – New England Region
Mortgage sales leader expanding Capital Bank Home Loans, Capital Bank’s nationwide mortgage lending division, across New England. Recruiting teams and driving regional production growth.
Area Sales Manager
Area Sales Manager expanding Capital Bank Home Loans’ Midwest mortgage business. Recruiting mortgage leaders, driving regional production, and building market relationships.
Area Sales Manager – Mid-Atlantic Region
Remote mortgage sales leader expanding Capital Bank Home Loans across Maryland, Washington DC, and Virginia. Recruiting branch managers and originators while driving regional growth and production.
Sales Representative IV
Technical Sales Representative selling EnerSys battery and stored-energy products. Managing customer relationships, technical proposals, and territory growth through Salesforce CRM.