AI Evaluator
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
AI Evaluator analyzing data and reviewing AI-generated responses for Gramian Consultancy’s global AI training projects. Providing structured feedback on accuracy, reasoning, and quality.
Responsibilities:
- Analyze and evaluate a variety of data and AI-generated content
- Annotate information according to project-specific guidelines
- Review AI outputs for accuracy, relevance, clarity, and quality
- Identify errors, inconsistencies, weak reasoning, and missing information
- Provide clear written feedback and recommendations
- Complete problem-solving and reasoning tasks across different subject areas
- Follow detailed project guidelines and evaluation criteria consistently
- Participate in quality-assurance and review processes
- Communicate findings and issues clearly with the project team
- Maintain high attention to detail across assigned tasks
Requirements:
- Excellent written and spoken English
- Strong critical-thinking and analytical skills
- Ability to evaluate information and identify logical or factual problems
Similar Jobs
AI Safety Expert, English – Odia
AI red teaming expert probing Mercor-trained frontier models for jailbreaks, bias, and misuse. Generating reproducible safety data and vulnerability reports for AI labs and enterprise customers.
AI Evaluator
AI Evaluator analyzing data and reviewing AI-generated content for Gramian Consultancy’s global AI training projects. Providing structured feedback on accuracy, reasoning, and quality.
Software, AI, IT, Data Evaluator
AI Evaluator reviewing documents, spreadsheets, and slide decks for accuracy and domain quality. Helping Mercor’s AI lab partners train and enhance frontier models.
Director, Information Technology – Enterprise AI
Director leading IT, cybersecurity, and enterprise AI strategy for Vehlo, an auto repair technology and financial solutions provider. Governing security, AI platforms, infrastructure, and technology operations.
Senior Director Analyst – Integration and APIs, AI
Senior Gartner analyst shaping AI-era integration and API research. Advising enterprise leaders on MCP, AI gateways, and integration strategy.
Senior Director, Analyst – Tech CEO Business and Strategy Advisor on AI Orchestration
Senior Gartner analyst advising Texas-based technology CEOs on AI Orchestration growth, market dynamics, and innovation. Creating research, frameworks, forecasts, and presentations for Gartner clients and events.
Senior Director, Analyst – Healthcare Providers, RCM & Healthcare AI Technology
Senior Director shaping Gartner insights on healthcare providers' U.S. revenue cycle management, coding, and AI technologies. Advising executives and clients through research, presentations, and strategic guidance.
AI Safety Red Teamer
AI Safety Red Teamer stress-testing frontier AI models for Mercor, which partners with AI labs to train and enhance AI systems. Identifying vulnerabilities and improving model safety.
Bilingual Dutch STEM Expert – AI Safety
PhD STEM expert evaluating Dutch-language scientific AI responses for Mercor. Writing specialist prompts and annotating model safety, accuracy, and helpfulness.
Bilingual Portuguese Generalist Expert – AI Safety
Bilingual Portuguese Generalist evaluating sensitive-topic AI model safety for Mercor. Writing prompts, classifying conversations, and flagging adversarial patterns.
Bilingual Danish STEM Expert – AI Safety
Bilingual Danish PhD STEM expert writing prompts and evaluating scientific responses. Improving AI safety for Mercor’s frontier-model training projects.
Bilingual German Generalist Expert – AI Safety
Bilingual German Generalist evaluating sensitive-topic AI behavior for Mercor’s frontier-model training projects. Writing German prompts, classifying conversations, and flagging adversarial patterns.
Bilingual Korean Generalist Expert – AI Safety
Bilingual Korean Generalist Expert evaluating sensitive-topic handling in Korean for Mercor’s frontier AI training projects. Writing prompts, classifying conversations, and flagging adversarial patterns.
Bilingual Danish Generalist Expert – AI Safety
Bilingual Danish Generalist Expert evaluating sensitive AI-model behavior for Mercor. Writing prompts, classifying conversations, and flagging adversarial patterns in frontier-model training projects.
Bilingual Russian Generalist Expert – AI Safety
Bilingual Russian Generalist evaluating sensitive-topic prompts and conversations for Mercor. Applying language expertise to train safer frontier AI models.
Bilingual Croatian STEM Expert – AI Safety
Bilingual Croatian PhD scientist evaluating scientific AI responses and writing safety prompts. Supporting Mercor’s frontier-model training through expert technical annotation.
AI Safety Red Teamer
AI Safety Red Teamer identifying vulnerabilities in frontier AI models for Mercor’s AI lab partnerships. Designing adversarial prompts and evaluating high-risk model behavior.
AI Safety Expert – English, Portuguese
AI safety red teamer probing conversational models for jailbreaks, prompt injections, and systemic risks. Producing human-data artifacts that help Mercor customers build safer frontier AI.
AI Safety Experts – English, Bengali
AI safety red team expert probing Mercor’s partner AI models for vulnerabilities. Generating reproducible adversarial data, reports, and attack cases to improve model safety.
AI Safety Red Teamer
AI Safety Red Teamer stress-testing frontier AI models for Mercor’s AI training partnerships. Identifying vulnerabilities and improving model alignment, robustness, and safety.