Mercor Research Fellowship – APEX
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Research fellow building and validating APEX benchmarks for Mercor, an AI data and enterprise evaluation company. Designing evaluations, testing frontier models, and publishing results.
Responsibilities:
- Propose and scope a new benchmark or evaluation technique in an under-covered APEX domain or a meaningfully harder version of an existing one
- Design task specifications and grading rubrics with Mercor’s vetted domain experts
- Build and validate benchmarks through task pilots, scoring calibration, and stress-testing for contamination and gameable shortcuts
- Run frontier models against benchmarks and analyze failure modes
- Publish results as a paper, open dataset, APEX leaderboard, or methodology adopted internally
- Partner with Mercor’s research and engineering teams to incorporate findings into APEX’s public benchmark family
- Work directly with the APEX research team and access enterprise evaluation problems from Fortune 500 and frontier-lab partners
Requirements:
- Genuine interest in evaluation as a research discipline
- Background in CS, ML, statistics, or an adjacent field such as measurement, psychometrics, HCI, or social science
- Specific, well-scoped idea for a benchmark or evaluation technique to build
- Comfortable working in a startup environment with fast iteration and less hand-holding than an academic lab
- Able to commit at least 20 hours/week for the duration of the fellowship
- Minimum commitment of 30 hours/week; full-time preferred
- Bonus: experience with agentic evaluation, RL environments, or domain expertise in law, finance, medicine, or a scientific field
- Must submit a specific benchmark or evaluation-methodology proposal
Benefits:
- Unlimited API credits
- Dedicated budget for GPU compute
- Paid expert/human-data time
- Weekly 1:1 mentorship with a member of the APEX research team
- Regular access to the broader research organization
- Access to frontier model APIs
- Access to Mercor’s internal evaluation infrastructure
- Access to real enterprise evaluation problems from Mercor’s customers, where appropriate
- Optional desk in Mercor’s San Francisco office
- Introductions to Mercor’s network of researchers across frontier labs and academia
- Standout fellows considered for a full-time offer on the APEX research team at the end of the fellowship
Similar Jobs
Research Assistant I
Part-time Research Assistant translating and disseminating UAMS health research findings. Developing evidence-based websites, reports, presentations, and community-facing materials.
Research Assistant – Infectious Diseases
Part-time remote research assistant supporting WashU’s infectious-disease and global public-health research. Conducting assays, analyzing data, coordinating studies, and supporting project operations.
Senior Research Scientist, Information Science – Extended Temporary
Senior Research Scientist developing AI/NLP systems for scientific feasibility assessment at the University of Arizona. Leading reproducible research pipelines, empirical studies, publications, and student collaboration.
Senior Researcher, Mixed Methods
Senior Researcher advancing decarbonization through behavioral and mixed-methods research at GoodPower. Designing experiments, evaluating interventions, and translating evidence into stronger energy-transition programs.
Senior Applied Scientist – Cyber Defense
Senior Applied Scientist building and deploying AI agents for NVIDIA’s cybersecurity operations. Evaluating models, optimizing agent systems, and automating detection, investigation, and response.
Senior Applied Research Scientist – Computational Geometry, Meshing
NVIDIA applied researcher developing computational geometry, meshing, and discretization algorithms. Advancing GPU-based CAD-to-simulation workflows across engineering, semiconductor, and scientific computing.
Research Scientist, PhD New Grad
Research Scientist building and deploying machine-learning models for SentiLink’s identity verification and fraud-risk products. Researching fraud patterns, engineering features, and delivering production code and analyses.
Applied Scientist, PhD New Grad
Applied Scientist building production fraud-detection models for SentiLink’s identity-risk solutions. Researching fraud, engineering features, and deploying real-time machine-learning systems.
Scientifique Principal des Données – IA
Senior AI Data Scientist advancing data and AI transformation for Valtech, an experience innovation company. Remote role limited to candidates in Ontario or Québec, requiring English fluency in Québec.
R&D Scientist III, Algorithm Developer
R&D Scientist developing and validating NGS analysis algorithms for molecular diagnostics and genomics. Documenting analytical methods and supporting assay development, quality, and regulatory needs.
Senior Market Researcher
Senior Market Researcher leading quantitative and mixed-method research at HubSpot. Collaborating with teams to derive insights for product strategies and market opportunities.
Research Scientist I, Pediatrics
Research Scientist I in Pediatrics at Medical College of Wisconsin, supervising research projects and collaborating on studies. Involves complex procedures, statistical analyses, and laboratory management.
Senior UX Designer, Researcher
Senior UX Designer/Researcher at Ad Hoc enhancing veteran digital services through user experience improvements. Collaborating with cross-functional teams to design impactful solutions and mentor junior designers.
Policy Research Fellow
Policy Research Fellow at the Hindu American Foundation researching extremism and national security. Conducting policy analysis and writing reports to support advocacy efforts.
Senior Principal Software Scientist
Senior Principal AI Scientist designing and training transformer models at Cerence AI. Leading AI innovations for the automotive industry with a focus on generative AI and large-scale systems.
Staff Research Scientist – Reinforcement Learning
Staff Research Scientist leading simulation environments and post-training AI solutions at Centific. Collaborating with data scientists and engineers to produce high-quality AI applications for enterprise workflows.
Principal Research Scientist
Principle Research Scientist at Syneos Health collaborating in drug development. Leading innovative problem-solving initiatives with a focus on customer success.
Senior UX Researcher
Senior UX Researcher leading qualitative and quantitative research to influence product strategy for Financial Reporting solutions at Workiva. Collaborating with cross-functional teams to shape AI-enabled experiences.
Research Assistant
Research Assistant analyzing conflict data for Mozambique at ACLED. Collaborating with the Southeast Africa Senior Analyst on data and literature analysis.
Applied Scientist
Applied Scientist leveraging large datasets to inform pricing and distribution decisions at Juul Labs. Collaborating with stakeholders to evaluate promotions and build forecasting models.