Social Workers, AI Conversation Evaluation
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
AI conversation evaluator reviewing advisory interactions for Gramian Consultancy, an IT professional-services and engineering talent-solutions consultancy. Identifying missing context and improving AI recommendations through structured qualitative feedback.
Responsibilities:
- Review AI conversations involving interpersonal, advisory, and everyday decision-making scenarios
- Assess whether the AI gathered sufficient context before making recommendations
- Identify important missing information and suggest relevant follow-up questions
- Determine whether additional context could materially change the recommendation
- Distinguish between load-bearing information and non-essential details
- Conduct structured qualitative reviews using defined evaluation guidelines
- Document findings clearly in spreadsheets, annotation tools, or labeling platforms
- Apply consistent judgment across cases and flag ambiguous examples
- Collaborate with research and quality teams to improve evaluation standards
- Participate in calibration and quality-review activities
Requirements:
- Background in psychology, social work, counseling, communication, qualitative research, HCI, conflict resolution, or related advisory disciplines
- MSW with LCSW or equivalent social-work license; or MA/MS with LMFT, LPC, LCPC, or equivalent counseling license; or PhD/PsyD in specified psychology fields; or MA/PhD in specified related fields
- Relevant professional credential combined with at least 5 years of direct advisory, coaching, mediation, consulting, or client-facing experience
- At least 3 years of relevant professional practice
- Experience gathering missing context through interviews, assessments, consultations, or advisory conversations
- Ability to distinguish critical information from non-essential context
- Strong written English and qualitative analytical skills
- Strong attention to detail and ability to follow detailed labeling guidelines
- Full-time commitment of 40 hours per week with at least 4 hours of PST overlap
- Successful completion of a short online test and delivery review
Similar Jobs
Licensed Social Worker, LSW
Licensed social worker delivering remote outpatient telehealth mental health care through Brave Health. Serving Medicaid, Medicare, and commercial-plan patients across the lifespan.
Licensed Master Social Worker, LMSW
Brave Health LMSW providing outpatient mental health treatment through telehealth. Delivering assessments, treatment planning, and psychotherapy to diverse patients remotely.
Licensed Clinical Social Worker, Spanish Speaking
Licensed therapist providing Spanish-speaking outpatient telehealth care through Brave Health, expanding access to mental health treatment. Conducting assessments, diagnosing conditions, and developing treatment plans remotely.
Licensed Clinical Social Worker
Licensed Clinical Social Worker supporting Bay Area home health patients and families. Providing counseling, crisis intervention, care coordination, and discharge support.
Supervising Clinician, LCSW-S / LPC-S – Supervision Only
Remote LCSW-S/LPC-S supervising associate therapists for Impireum Psychiatric Group. Ensuring documentation quality, compliance, clinical development, and audit readiness without direct patient care.
Behavioral Health Utilization Management Clinician, LCSW, LPC, LMFT, RN, LSATP
Behavioral health utilization clinician reviewing substance-abuse services for Sentara’s Virginia health plan. Applying medical-necessity criteria and payer regulations to authorize care.
Perinatal Social Worker
Perinatal Social Worker supporting public health programs by evaluating maternal healthcare delivery and analyzing social determinants of health. Responsible for developing interview protocols and training for clinical review teams.
No more related jobs available.