Senior Test Engineer
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Senior Test Engineer validating AI-powered patient summarization for VetsEZ’s federal healthcare modernization project. Developing automated testing, interoperability validation, and AI quality controls for clinical software.
Responsibilities:
- Develop comprehensive test plans, test cases, and acceptance criteria for AI-powered patient summarization capabilities
- Create traceability between business requirements, clinical data standards, and test scenarios
- Design test coverage across clinical document types, including CCDs, Discharge Summaries, Progress Notes, and History & Physical documents
- Develop positive, negative, boundary, and edge-case testing scenarios for AI-enabled clinical applications
- Analyze and validate C-CDA XML documents, including headers, sections, templates, and clinical entries
- Verify accurate extraction and summarization of clinical information, including medications, allergies, laboratory results, procedures, care plans, and diagnoses
- Ensure AI-generated summaries accurately reflect source clinical documentation and maintain traceability to original clinical sources
- Perform functional, integration, system, regression, performance, and user acceptance testing
- Develop and maintain automated test suites for document ingestion, AI summarization, APIs, and user interfaces
- Test CDS workflows and patient-context-driven launch scenarios across multiple data sources and patient encounters
- Support CI/CD pipelines through automated quality gates and continuous testing
- Build automated validation frameworks for AI-generated summaries
- Evaluate AI-generated content for clinical completeness, consistency, usability, accuracy, and reliability
- Identify omissions, inaccuracies, hallucinations, and other AI-generated defects that could impact clinical workflows
- Validate AI guardrails, monitoring capabilities, auditability, and quality controls
- Participate in defect triage, root-cause analysis, and release-readiness activities
- Create test documentation, defect reports, traceability matrices, and quality assessments
- Collaborate with software engineers, solution architects, clinicians, product owners, cybersecurity teams, and government stakeholders
- Leverage AI-assisted tools to accelerate test development, test automation, regression testing, defect analysis, and quality assurance activities
- Take on additional tasks and responsibilities as needed to support team objectives and ensure project success
Requirements:
- Bachelor's degree in Computer Science, Information Technology, Health Informatics, Engineering, or a related technical discipline, or equivalent experience
- Minimum of 5 years of experience testing healthcare software applications
- Strong knowledge of Consolidated Clinical Document Architecture (C-CDA) standards, including clinical document structure, sections, templates, and entries
- Hands-on experience reading, validating, and troubleshooting XML-based healthcare data
- Experience with healthcare interoperability standards and clinical data exchange
- Experience testing Clinical Decision Support (CDS) applications and workflows
- Familiarity with SMART on FHIR, CDS Hooks, REST APIs, and related interoperability technologies
- Experience with test automation frameworks, API testing tools, and defect management platforms
- Experience developing and maintaining automated testing solutions within Agile and DevSecOps environments
- Experience with AI-assisted testing tools and techniques for test case generation, automation development, regression testing, defect analysis, and quality assurance
- Familiarity with Generative AI and Large Language Models (LLMs), including using AI to create and maintain automated test scripts, validation frameworks, and test data
- Experience validating AI-generated outputs and establishing quality controls for accuracy, consistency, repeatability, and reliability
- Strong analytical, troubleshooting, and problem-solving skills
- Excellent written and verbal communication skills
- Experience supporting Department of Veterans Affairs (VA) healthcare systems, including JLV or similar clinical viewer applications
- Knowledge of USCDI and healthcare interoperability requirements
- Familiarity with SNOMED CT, LOINC, RxNorm, and ICD
- Experience validating AI/ML-enabled healthcare solutions
- Experience with AI-powered testing platforms such as GitHub Copilot, Microsoft Copilot, Testim, Functionize, Mabl, or similar tools
- Experience creating autonomous or semi-autonomous AI testing workflows
- Understanding of prompt engineering and AI evaluation methodologies for software quality assurance
- Knowledge of accessibility and usability testing within healthcare environments
- Experience working within Agile, DevSecOps, and cloud-native development environments
- Exposure to healthcare data analytics and modernization initiatives within federal healthcare agencies
- U.S. Citizenship is required
- All selected candidates must successfully complete a background check
- Ability to obtain and maintain a Government/Public Trust clearance, including required fingerprinting, when required for the position
- Sponsorship is not available
Benefits:
- Medical/Dental/Vision
- 401k with Employer Match
- PTO + Federal Holidays
- Corporate Laptop
- Training Opportunities
- Remote Opportunity
Similar Jobs
Senior Automation Test Engineer, Selenium – Java
Senior automation test engineer building Selenium Java QA solutions for Accesa, a technology company serving major retail, manufacturing, finance, and banking brands. Leading scalable automation, CI/CD quality, and cross-functional testing.
Senior General Test Engineer
Senior QA engineer automating UI, API, and performance testing for a global mining company through ELEKS. Developing test scripts, documentation, metrics, and quality assurance processes.
AI SDET – Software Development Engineer in Test
AI SDET testing AI agents for safety, compliance, and accuracy at CI&T, an enterprise AI and technology transformation company. Managing evaluation harnesses, regulatory quality gates, shadow validation, and adversarial automated testing.
Senior Software Engineer – Test
Senior SDET improving quality for Replicant’s voice AI customer-service platform. Building AI-driven testing, automation frameworks, analytics, and CI/CD quality gates for scalable releases.
Senior Test Automation Engineer, Python
Senior QA engineer testing Python automation, low-code applications, and AI agents for Avenga. Defining strategies, validating outputs, tracking defects, and owning release sign-off.
Test Automation Engineer
Test Automation Engineer building UI, API, and integration testing frameworks for GDIT. Improving software quality for U.S. government, defense, and intelligence missions.
Lead QA Automation Engineer
Lead QA automation for Perch, a digital mortgage platform helping Canadians build wealth through real estate. Setting quality standards, managing QA engineers, and maintaining automated regression coverage.
Automation Test Engineer
Remote Test Automation Engineer building UI, API, and integration test frameworks for a U.S. Government client. Improving reliability, coverage, and defect prevention across modern application stacks.
QA Engineer – Automation, Functional
QA Engineer automating and validating clinical-grade cardiac monitoring software at RhythmScience. Building Playwright coverage, testing web/mobile apps, and improving release quality.
QA Test Engineer
QA Test Engineer for ALTEN Mexico, an engineering and IT services company, testing web applications, APIs, and databases. Designing test plans, automating tests, tracking defects, and validating releases.
Automation Test Engineer
Automation Test Engineer developing cloud-based automated testing for GDIT’s U.S. government case-management modernization. Building QA frameworks, scripts, and CI/CD integrations for enterprise applications.
Middle Test Engineer
Middle Test Engineer testing a PDF reader with document-signing capabilities for ELEKS in Ukraine. Executing tests, managing test artifacts, and reporting progress to stakeholders.
Senior QA Automation Engineer, Playwright/TypeScript
Senior QA Automation Engineer building Playwright/TypeScript coverage for a US telehealth platform. Integrating automated testing across healthcare workflows, APIs, integrations, and AWS CI/CD.
Senior Test Engineer
Senior Test Engineer defining testing strategies and automating quality assessments for software serving European institutions. Remote within the EU, collaborating with development and client IT teams.
QA Automation Analyst – Testing Funcional y Automatización
QA Automation Analyst in a consulting firm providing custom technological solutions. Designing testing plans and collaborating with development teams to ensure application quality.
Senior Hardware Test Development Engineer, Python
Hardware Test Development Engineer ensuring quality and reliability of hardware products for Jabil. Design, develop, and maintain test systems and software in a collaborative team environment.
Senior Test Automation Engineer – ConnectWise
Senior Test Automation Engineer enhancing test automation frameworks for ConnectWise. Focused on migrating to new platforms with CI/CD integration and team collaboration.
Test Development Engineer Intern
Test Development Engineer Intern at Bybit participating in full product testing lifecycle. Collaborating cross-functionally and ensuring product quality through various testing methods.
Senior Technical Account Manager – Automation Test Engineer
Technical Account Manager facilitating technical onboarding and customer success in visual AI test automation. Collaborating with engineering teams and driving technical strategies for customer engagement.
Software Engineer, Test
Software Engineer responsible for automated test coverage for ServiceTitan's Dispatch Engineering team. Building tests with Playwright for a high-impact, AI-forward squad.