Clinical Evaluator, AI Products – Contract
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Clinical Evaluator testing Headway’s AI mental-health products for clinical accuracy and safety. Refining evaluation rubrics and guiding product launch decisions.
Responsibilities:
- Review AI tool output across Headway's patient and provider products for clinical accuracy, safety, and appropriateness
- Apply evaluation rubrics, improve them, and document clinical reasoning when rubrics fall short
- Participate in calibration sessions and inter-rater reliability rounds with peer evaluators
- Collaborate with the Clinical Product team to translate clinical feedback into rubric or output changes
- Label datasets and review new product use cases before launch
- Complete review and feedback work in Headway's designated systems while adhering to HIPAA and privacy/confidentiality requirements
Requirements:
- Current licensed therapist or professional counselor with an independent, unrestricted license
- 3+ years of direct therapy experience
- Strong grasp of risk assessment, safety planning, and documentation standards
- Ability to critically evaluate AI model output and provide structured feedback
- Ability to explain clinical reasoning clearly in writing
- Comfortable working through ambiguous edge cases with peer evaluators
- Comfortable adapting to healthcare technology platforms and evolving workflows
- Experience with AI chart reviews, quality audits, or clinical documentation standards is a plus
- Must set up or already have an LLC or S-Corp
- Must be authorized to work in the United States
Benefits:
- Comprehensive and competitive total rewards package
- Robust health and wellness benefits
- Retirement savings
- Meaningful ownership opportunities through equity
- Reasonable accommodations for persons with disabilities
Similar Jobs
AI Harness Engineer
AI Harness Engineer building reliable AI tools, agent harnesses, and P2P systems for Tether’s local AI assistants. Advancing QVAC within Tether’s blockchain-powered digital finance ecosystem.
AI Harness Engineer
AI Harness Engineer building agent harnesses and P2P technology for Tether’s local AI products. Delivering reliable tools, modules, and scalable integrations across model types.
AI Safety Red Teamer
AI Safety Red Teamer adversarially testing frontier AI models for Mercor, which partners with AI labs to train and enhance frontier systems. Identifying vulnerabilities across high-risk domains.
AI Prompt & Workflow Specialist
AI Prompt & Workflow Specialist designing and optimizing generative AI solutions for ExamWorks’ healthcare services operations. Translating business requirements into tested prompts, agents, and automation workflows.
Quality Control Specialist – AI/ML, Ukrainian Language
Quality control specialist reviewing Ukrainian AI speech transcripts for TSMG. Correcting display-text formatting, punctuation, capitalization, and normalization issues.
AI Enablement Specialist
AI Enablement Specialist evaluating and implementing AI tools, automated workflows, and process improvements for Wilson Sonsini’s legal practices. Leading pilots, adoption, training, and cross-functional innovation initiatives.
Head of Teaching – AI & Future Skills
Head of Teaching leading nationwide remote education operations for an EdTech scale-up. Shaping AI-focused curricula, teaching quality, processes, and scalable learning programs.
Clinical Review RN, Oncology AI Review
Clinical Review RN validating AI-driven oncology workflows for Natera, a global cfDNA testing company. Reviewing complex cases, payer policies, and AI classifications.
AI Safety Red Teamer
AI Safety Red Teamer stress-testing frontier AI models for Mercor, which partners with AI labs to train and enhance AI systems. Uncovering jailbreaks, hallucinations, and policy failures.
AI Safety Red Teamer
AI Safety Red Teamer stress-testing frontier AI models through adversarial prompts. Identifying vulnerabilities and improving model alignment, robustness, and safety for Mercor’s AI lab partners.
AI Safety Red Teamer
AI Safety Red Teamer identifying vulnerabilities in frontier AI systems for Mercor's partner AI labs. Designing adversarial prompts and evaluating high-risk model behavior.
Scientific Language & AI Adaptation Contributor – Korean, Japanese, Arabic
Scientific language contributor helping Maxibook, an AI education startup, adapt English scientific content across Korean, Japanese, and Arabic. Validating AI outputs for conceptual accuracy, naturalness, and academic credibility.
AI Safety Red Teamer
AI Safety Red Teamer stress-testing frontier AI systems for Mercor. Designing adversarial prompts and documenting vulnerabilities to improve model safety and alignment.
Research Staff, Voice AI Foundations
Research Staff pioneering latent-space models, neural audio codecs, and speech systems at Deepgram. Advancing scalable, efficient voice AI for real-time human-machine interaction.
Research Staff – Voice AI Foundations
Research Staff developing latent-space models for Deepgram’s voice AI platform. Advancing neural audio codecs, generative speech, multimodal systems, and hardware-efficient inference.
Research Staff, Voice AI Foundations
Research Staff pioneering latent-space models for codecs, speech generation, and multimodal voice AI. Advancing Deepgram’s real-time speech platform through scalable, efficient foundational research.
AI Operations Engineer
AI Operations Engineer building and supporting enterprise AI agents, chatbots, and integrations. Enabling secure, governed AI operations for Turtle’s electrical distribution business.
AI Evaluations Engineer – School of Government and Policy
Remote AI evaluations engineer managing sensitive data, audits, queries, and reports. Supporting Johns Hopkins’ School of Government and Policy with data processes and user training.
Business Logic Expert – AI Community
Business Logic Expert reviewing AI-generated business content for professional accuracy and etiquette. Helping TELUS Digital improve AI systems through remote, flexible freelance work.
Head of Sales – Custom Software & AI
Head of Sales für Sidestreams individuelle Software- und KI-Lösungen. Verantwortung für Neukundengeschäft, Teamführung, Abschlüsse und Upsell in Deutschland.