Healthcare Subject Matter Expert – AI Model Evaluation
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Healthcare SMEs evaluating clinical accuracy, reasoning, and safety in AI-generated medical content for Volga Partners. Project-based remote assignments for physicians, nurse practitioners, and physician assistants.
Responsibilities:
- Evaluate AI-generated medical content for clinical accuracy, completeness, relevance, and consistency with accepted medical standards
- Analyze AI-generated clinical reasoning, assessments, differential diagnoses, and treatment recommendations
- Identify inaccurate medical information, unsupported conclusions, missing clinical considerations, and potential patient safety concerns
- Validate medical information using authoritative medical literature, established clinical guidelines, and evidence-based healthcare practices
- Provide expert feedback, explain findings, identify improvements, and recommend clinically appropriate corrections
- Apply evaluation guidelines, scoring criteria, and project-specific quality standards
- Complete independent evaluations using designated online platforms within project timelines
- Contribute clinical expertise to the development and evaluation of advanced AI models
- Work on assignments whose schedules, workloads, and availability are determined by project needs
Requirements:
- Minimum of three years of relevant professional clinical experience in an applicable specialty
- Recognized medical degree, advanced-practice qualification, or equivalent professional credential appropriate to the specialty and country
- Demonstrated experience applying clinical judgment, diagnostic reasoning, or evidence-based medical practices
- Strong understanding of medical terminology, clinical guidelines, and healthcare standards relevant to the candidate's specialization
- Ability to critically evaluate complex medical information and distinguish accurate clinical reasoning from unsupported conclusions
- Strong analytical, research, and problem-solving abilities
- Excellent written English communication skills, including the ability to clearly explain clinical findings and professional assessments
- Comfort using digital platforms, web-based applications, and online research tools
- Ability to work independently, follow detailed instructions, and meet established quality standards
- Must be comfortable with and understand that this is a task-based role
- Current or previous professional licensure or registration relevant to the candidate's specialty is preferred
- Experience reviewing medical records, clinical research, medical guidelines, or healthcare quality assessments is preferred
- Familiarity with U.S. healthcare standards or clinical practices in the United Kingdom, Canada, Australia, or New Zealand is preferred
- Experience evaluating clinical documentation, medical literature, or complex healthcare cases is preferred
- Previous exposure to AI model evaluation, medical content review, clinical research, or healthcare technology is preferred
Benefits:
- Remote work from an eligible location
- Project-based freelance work
- Flexible, variable schedule with potential weekday and weekend assignments
- Opportunity to contribute to advanced AI model evaluation
- No AI background required
- Supplemental-income opportunity
Similar Jobs
Healthcare Subject Matter Expert – AI Model Evaluation
Healthcare professionals evaluating AI-generated medical content for Volga Partners. Applying clinical judgment to improve the accuracy and reliability of healthcare AI models.
AI Evaluation Engineer
AI Evaluation Engineer building statistical evaluation, CI/CD testing, and observability infrastructure for Scopic’s production AI agents. Designing adversarial, multi-turn, and structured-output test systems.
English AI Safety Specialist
AI Safety Specialist reviewing English conversations and labeling AI responses. Helping Mercor’s AI lab and enterprise partners train and enhance frontier models.
Clinical Evaluator, AI Products – Contract
Clinical Evaluator testing Headway’s AI mental-health products for clinical accuracy and safety. Refining evaluation rubrics and guiding product launch decisions.
AI Harness Engineer
AI Harness Engineer building reliable AI tools, agent harnesses, and P2P systems for Tether’s local AI assistants. Advancing QVAC within Tether’s blockchain-powered digital finance ecosystem.
AI Harness Engineer
AI Harness Engineer building agent harnesses and P2P technology for Tether’s local AI products. Delivering reliable tools, modules, and scalable integrations across model types.
AI Safety Red Teamer
AI Safety Red Teamer adversarially testing frontier AI models for Mercor, which partners with AI labs to train and enhance frontier systems. Identifying vulnerabilities across high-risk domains.
AI Prompt & Workflow Specialist
AI Prompt & Workflow Specialist designing and optimizing generative AI solutions for ExamWorks’ healthcare services operations. Translating business requirements into tested prompts, agents, and automation workflows.
Quality Control Specialist – AI/ML, Ukrainian Language
Quality control specialist reviewing Ukrainian AI speech transcripts for TSMG. Correcting display-text formatting, punctuation, capitalization, and normalization issues.
AI Enablement Specialist
AI Enablement Specialist evaluating and implementing AI tools, automated workflows, and process improvements for Wilson Sonsini’s legal practices. Leading pilots, adoption, training, and cross-functional innovation initiatives.
Head of Teaching – AI & Future Skills
Head of Teaching leading nationwide remote education operations for an EdTech scale-up. Shaping AI-focused curricula, teaching quality, processes, and scalable learning programs.
Clinical Review RN, Oncology AI Review
Clinical Review RN validating AI-driven oncology workflows for Natera, a global cfDNA testing company. Reviewing complex cases, payer policies, and AI classifications.
AI Safety Red Teamer
AI Safety Red Teamer stress-testing frontier AI models for Mercor, which partners with AI labs to train and enhance AI systems. Uncovering jailbreaks, hallucinations, and policy failures.
AI Safety Red Teamer
AI Safety Red Teamer stress-testing frontier AI models through adversarial prompts. Identifying vulnerabilities and improving model alignment, robustness, and safety for Mercor’s AI lab partners.
AI Safety Red Teamer
AI Safety Red Teamer identifying vulnerabilities in frontier AI systems for Mercor's partner AI labs. Designing adversarial prompts and evaluating high-risk model behavior.
Scientific Language & AI Adaptation Contributor – Korean, Japanese, Arabic
Scientific language contributor helping Maxibook, an AI education startup, adapt English scientific content across Korean, Japanese, and Arabic. Validating AI outputs for conceptual accuracy, naturalness, and academic credibility.
AI Safety Red Teamer
AI Safety Red Teamer stress-testing frontier AI systems for Mercor. Designing adversarial prompts and documenting vulnerabilities to improve model safety and alignment.
Research Staff, Voice AI Foundations
Research Staff pioneering latent-space models, neural audio codecs, and speech systems at Deepgram. Advancing scalable, efficient voice AI for real-time human-machine interaction.
Research Staff – Voice AI Foundations
Research Staff developing latent-space models for Deepgram’s voice AI platform. Advancing neural audio codecs, generative speech, multimodal systems, and hardware-efficient inference.
Research Staff, Voice AI Foundations
Research Staff pioneering latent-space models for codecs, speech generation, and multimodal voice AI. Advancing Deepgram’s real-time speech platform through scalable, efficient foundational research.