Behavioral Health Expert – AI Safety, Model Evaluation

Posted 1ds ago

Employment Information

Industry

Education

Salary

Experience

Job Type

Location

Job expired or something wrong with this job?

Behavioral health experts evaluating sensitive AI conversations for leading AI labs. Defining safe, neutral responses and improving model judgment.

Evaluate conversations between users and AI models on relationship and family advice, emotional wellbeing, spiritual and metaphysical questions, and unconventional or unfounded beliefs
Assess whether model responses are neutral, appropriate and safe
Identify and document concerns such as excessive agreement, taking sides, reinforcing distorted or unfounded beliefs, moralizing, or overstepping into clinical or directive advice
Develop rubrics, guidelines and reference responses for balanced, supportive and appropriately bounded replies
Design test scenarios and conversations for sensitive, non-crisis topics
Collaborate with lab researchers and fellow experts to maintain consistent, calibrated and well-documented evaluation standards

Degree in psychology, counseling, social work, behavioral health, behavioral science, human services or a closely related field, or equivalent professional experience in the mental health field
3+ years of professional experience supporting people in a mental health, counseling or social services setting
Ability to remain neutral and nonjudgmental across diverse perspectives, relationships, belief systems and worldviews
Ability to explain the reasoning behind professional judgments
Working familiarity with sycophancy, cognitive distortions, healthy boundaries and client-centered approaches such as motivational interviewing
Ability to engage reliably for at least 20 hours per week during weekdays
Strong written communication skills and ability to deliver precise, well-structured written feedback
Clinical licensure is valued but not required
Nice to have: background in AI safety, applied ethics, trust and safety or content policy
Nice to have: experience in couples, family or relationship counseling
Nice to have: familiarity with spiritual care, religious or alternative belief communities, or the psychology of misinformation and conspiracy belief
Nice to have: prior experience evaluating, annotating or red teaming AI systems
Unable to support H1-B or STEM OPT candidates at this time