Senior AI Quality Engineer
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Senior AI Quality Engineer ensuring the reliability of AI-powered applications for national security. Designing testing frameworks and collaborating with cross-functional teams to enhance quality assurance.
Responsibilities:
- Participate across the AI solution lifecycle from requirements and design through development, testing, and deployment
- Build and maintain systems that exercise agentic workflows against realistic scenarios, edge cases, and failure conditions
- Define contracts, validation, and explicit failure behavior at integration points
- Create shared libraries, reference harnesses, and validation utilities that teams can reuse
- Architect agentic systems so correctness, safety, and expected behavior are enforced by design
- Participate in solution design to identify likely failure patterns in agentic systems
- Define and implement cost-conscious evaluation strategies for AI-enabled systems
- Work with cross-functional program teams to translate customer and mission needs into concrete evaluation criteria
- Communicate estimates, progress, and technical tradeoffs clearly
Requirements:
- 5+ years of professional software engineering experience
- Demonstrated experience building bespoke validation, evaluation, or testing frameworks for complex systems
- Data validation, schema enforcement, and contract testing experience in distributed or service-oriented architectures
- Experience treating quality, reliability, and correctness as core design concerns
- Experience defining, interpreting, and acting on quality signals to assess system readiness
- Hands-on experience designing, building, and operating AI-powered systems
- Solid understanding of Amazon Web Services and its native AI/ML services
- Practical experience integrating automation into CI/CD pipelines
- Working knowledge of security best practices relevant to regulated or mission-critical contexts
- Proven ability to work effectively with cross-functional teams.
Benefits:
- Paid Time Off
- Paid Company Holidays
- Medical, Dental & Vision Insurance
- Optional HSA and FSA
- Base and Voluntary Life Insurance
- Short Term & Long-Term Disability Insurance
- 401k Matching
- Employee Assistance Program
Similar Jobs
Quality Assurance Consultant, PV Support
Quality Assurance Consultant providing operational QA and Pharmacovigilance support for a global pharmaceutical company. Responsibilities include handling quality deviations, training, and compliance management.
Clinical Quality Assurance Agent
Conducting quality assurance audits across clinical processes to ensure compliance with healthcare standards. Collaborating with teams and providing insights for improvements while working remotely from South Africa.
Software Test Analyst
Software Test Analyst ensuring the quality and reliability of government web applications undergoing modernization. Collaborating on testing methodologies and implementing automated test suites.
Senior Manual QA Engineer
Senior Manual QA Engineer for IT company providing B2B services and tech solutions. Engaging in testing and supporting projects in the iGaming sector.
Associate QA Engineer, SAP MM
Associate QA Engineer executing quality assurance for SAP initiatives in Brazil. Collaborating with Brazil and India teams to ensure high-quality deliverables and testing alignment.
Associate QA Engineer, SAP FI
QA Engineer responsible for executing QA activities for SAP FI in Brazil. Collaborating with cross-regional teams and ensuring high-quality deliverables.
QA Content Engineer
QA Cyber Content Engineer validating and ensuring quality of cyber security learning platform content at TryHackMe. Collaborating with content developers and supporting community interactions.
Consultor Tester ERP, SAP
Consultant Tester SAP providing technological transformations for companies in Colombia. Designing and executing functional tests, ensuring quality and efficiency in deployment processes.
Automation Tester
Test Automation Engineer for CACI, supporting transportation/logistics operations for the Department of Defense. Involved in Agile development and automated testing practices.
Analista QA
Analista QA responsible for designing, executing, and reporting on test cases for Falabella projects. Collaborating with clients and project leaders in a remote work environment.
Automation Tester
QA Consultant specializing in transitioning test automation framework from Leapwork to Playwright. Requires strong expertise in Playwright and software testing principles.
Claims Quality Assurance Director
Director of Claims Quality at Aspire General Insurance Company leading claims quality assurance programs and cross-functional collaborations. Driving performance improvements and regulatory compliance in a fully remote role.
Senior Test Analyst
Testing Analyst in a Brazilian technology group, focusing on software testing and quality assurance. Engaging in functional tests and supporting internal PLM system calls.
Quality Assurance Engineer
Quality Assurance Engineer for testing software, ensuring quality for product development team and validating releases. Focus on bug detection and process improvement in a remote setup.
Salesforce QA Analyst – Environmental Health
Salesforce QA Analyst ensuring environmental health applications function correctly while supporting public health initiatives at Flourish and Thrive Labs.
QA Automation Developer, Salesforce
QA Automation Developer enhancing Salesforce customizations for mortgage financing technology. Engaging in test automation, system testing, and regression testing within Salesforce applications.
Backend QA Engineer
Backend QA Engineer testing APIs and databases to ensure reliability and performance at Air Apps. Collaborate with engineering teams to maintain high-quality standards across infrastructure.
Backend QA Engineer
Backend QA Engineer ensuring reliability, performance, and scalability of backend systems at Air Apps. Collaborating on API, database, and service integrity testing.
Backend QA Engineer
Backend QA Engineer at Air Apps to ensure reliability, performance, and scalability of backend systems along with testing APIs and databases.
Business Analyst, Tester
Test Engineer providing expertise in business process and system analysis and design at GDIT. Collaborating with teams to enhance DevSecOps processes and ensure testing compliance.