QA Engineer, Voice AI Agents
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
QA Engineer debugging voice AI calls, campaigns, and healthcare workflows for Arbiter’s care orchestration platform. Investigating failures, improving QA processes, and automating validation.
Responsibilities:
- Investigate reported and detected issues on live campaigns, including dropped calls, wrong agent responses, failed handoffs, incorrect scheduling outcomes, and data mismatches
- Trace problems across call recordings, transcripts, system logs, campaign configuration, and integration data to identify root causes
- Write clear, reproducible bug reports with severity, member impact, and supporting evidence
- Triage issues and route them to the appropriate owner
- Run pre-launch QA on campaigns and workflow changes, testing scripts, edge cases, escalation paths, and handoffs
- Perform ongoing conversation QA on live campaigns and flag trends
- Verify targeting, sequencing, disposition logic, and outcome capture
- Help build and maintain test cases, scoring rubrics, regression suites, and release checklists
- Automate repetitive checks such as transcript analysis, configuration validation, and outcome reconciliation
- Track quality metrics and report patterns to deployment, product, and engineering teams
- Work cross-functionally with engineers, deployment managers, and clinical and nurse operations staff
Requirements:
- 3 to 5 years in QA, support engineering, technical operations, or a similar role on a production software platform
- Strong debugging instinct
- Comfort reading logs, querying data with SQL, and working with APIs and JSON or YAML configuration
- Experience writing test plans and bug reports
- Clear written communication and precise documentation habits
- Good judgment on severity and prioritization
- Preferred: experience with voice AI, conversational AI, IVR, or contact center platforms
- Preferred: scripting ability in Python or similar
- Preferred: exposure to LLM evaluation, prompt testing, or conversation quality scoring
- Preferred: healthcare experience, especially payer or provider outreach, and familiarity with HIPAA and PHI handling
- Preferred: experience with observability tools or call analytics tools
Benefits:
- Meaningful equity
- Generous paid time off (PTO)
- Comprehensive health, vision, and dental insurance
- Life and disability insurance
- Simple IRA matching
- Professional development budget for conferences, courses, and certifications
- Wellness programs
Similar Jobs
Application Tester
Application Tester validating digital payment user journeys on mobile devices for RecruitGo, an Employer of Record connecting emerging-market talent with global clients. Reporting defects and feedback remotely from Spain.
Construction QA/QC Manager
Construction QA/QC Manager overseeing MEP quality, commissioning readiness, and turnover for Edged Data Centers’ mission-critical facilities. Ensuring reliable, compliant infrastructure delivery.
Product Tester, Books and Ebooks
Remote freelance reader-testing books and eBooks for Onix Media GmbH, a publishing house. Evaluating purchases, delivery, and content while providing customer feedback.
Test Analyst, QA
Analista de QA garantindo a qualidade de produto financeiro digital da FCamara. Testando apps iOS/Android, APIs, Backend e jornadas ponta a ponta.
Junior Test Analyst
Analista de QA estruturando testes manuais e automatizados para a nova plataforma de Renda Fixa da FCamara. Validando APIs, integrações Core Banking e critérios de aceite.
Senior Test Analyst
Analista QA sênior liderando automação e testes de uma nova plataforma brasileira de Renda Fixa. Validando integrações com Core Banking e aplicando IA Generativa.
Senior Specialist, Supplier Quality Engineer
Senior Supplier Quality Engineer qualifying suppliers and strengthening manufacturing quality for L3Harris aerospace and defense programs. Supporting missile production expansion through risk management, process control, and supplier development.
Bilingual Software Tester, CYARA Contact Centre, AWS Connect
Bilingual software tester validating French online applications and phone systems for PLATO, a Canadian Indigenous-owned software testing services company. Testing CYARA Contact Centre and AWS Connect systems for finance-sector clients.
QA Engineer
QA Engineer testing Burny Games’ mobile puzzle games across platforms. Improving QA workflows with AI, validating monetization systems, and supporting reliable releases.
French Quality Assurance, Reviewer
French QA reviewer checking AI data collection submissions for quality and compliance. Identifying errors, documenting findings, and supporting Clickworker’s quality standards remotely.
Senior Fullstack Software Engineer, Quality Platform
Senior Fullstack Software Engineer building AI-powered test automation and quality platforms for Airbnb’s global stays and experiences marketplace. Leading robust testing across web, backend, and CI/CD systems.
System Analyst, Tester
Systems analyst and tester supporting Koniag Government Services’ federal client modernization project. Testing an AI-enabled application through automation, API validation, Agile collaboration, and defect management.
QA Engineer
QA Engineer ensuring reliable, high-performance testing for Gorilla’s energy margin intelligence platform. Automating tests and supporting quality releases for energy retailers.
Software Quality Assurance Manager
Quality Engineering Manager leading AI-enabled test automation for World Travel Holdings, a cruise and leisure travel company. Modernizing frameworks, CI/CD quality gates, and enterprise testing practices.
Software Quality Engineer – M/F/D
Software Quality Engineer securing PHP and Node.js software quality at secupay, a German omnichannel payments provider. Establishing code standards, CI/CD checks, and security reviews.
Mystery Shopper – Quality Assurance
Shop Auditor prüft Markenstandards, Warenpräsentation und Promotionen in Schweizer Verkaufsstellen. Dokumentiert Audits per Smartphone und gibt Filialpersonal konstruktives Feedback.
Quality Assurance Technician
Quality Assurance Technician reviewing work, data, and processes for accuracy and compliance. Documenting defects, tracking trends, and supporting corrective actions remotely.
Engineering Manager, QA
QA Engineering Manager leading distributed testing for Scribe’s workflow-capture products. Driving reliable enterprise rollouts across desktop apps and browser extensions.
Senior Quality Assurance Engineer
Senior QA Engineer validating RF, RS485, and IoT communication in Kidde Global Solutions’ security products. Supporting automation, EN 50131 compliance, and robust device-to-backend testing.
Clinical Quality Assurance Coordinator
Clinical Quality Assurance Coordinator reviewing medical peer-review reports for MES Solutions, a provider of independent medical examination and peer-review services. Ensuring evidence-based quality, regulatory compliance, and accurate documentation.