QA Engineer
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
QA Engineer designing automated tests for GenAI, LLM, Agentic AI, and RAG systems. Integrating Python and pytest testing into CI/CD pipelines for Weekday’s client.
Responsibilities:
- Define and implement test plans for GenAI, LLM, Agentic AI, and RAG systems
- Design and maintain automated test suites using pytest and Python
- Develop and execute API tests for model endpoints, data pipelines, and microservices
- Perform functional, regression, and performance testing of AI services
- Integrate testing into CI/CD pipelines with tools like Jenkins, GitHub Actions, or GitLab CI
- Analyze model output quality, bias, and fairness metrics and document findings
- Collaborate with ML teams to create test harnesses for new model releases
- Create and maintain test data sets, fixtures, and mock services
- Report defects, track resolution, and validate fixes to ensure quality standards
- Mentor junior testers and promote best practices in AI QA
- Work closely with data scientists, ML engineers, and product managers to validate Agentic AI and RAG workflows
- Monitor performance metrics, detect anomalies in generated outputs, and maintain documentation and dashboards
- Drive adoption of scalable QA practices across the organization
Requirements:
- 4+ years of QA engineering experience in AI or data science environments
- Strong proficiency in pytest, Python, and API testing frameworks (e.g., Requests, RestAssured)
- Hands-on experience with GenAI, LLM, Agentic AI, and RAG testing methodologies
- Familiarity with CI/CD pipelines and automation tooling (Jenkins, GitHub Actions)
- Deep understanding of AI ethics, bias mitigation, and model explainability
- Excellent analytical and problem-solving skills with a detail-oriented mindset
- Ability to translate complex AI concepts into clear, testable requirements
- Strong communication skills for cross-functional collaboration
- Proven track record of delivering high-quality releases on time
- Passion for staying current with emerging AI technologies and testing trends
Similar Jobs
LQA Game Tester, Czech
Freelance Czech LQA tester checking localized games for a global video game development and services provider. Finding language, cultural, formatting, and platform-compliance issues.
Lead QA Engineer
Lead QA Engineer owning web, API, and mobile quality for Backcountry’s outdoor e-commerce business. Building Playwright automation, CI/CD validation, and AI-assisted testing.
Head of Quality Engineering
Head of Quality Engineering leading global quality engineering, DevOps, and continuous testing for ABB’s industrial automation platform. Driving product verification, compliance, and QE transformation across PCP.
Senior QA Analyst
Analista QA Sênior garantindo a qualidade dos produtos digitais da cVortex. Definindo planos de teste e validando aplicações, APIs e integrações E2E.
Senior Quality Engineer
Senior Quality Engineer creating automated and manual tests, analyzing quality metrics, and leading root-cause resolution. Remote role based in Pune-Baroda, India.
Quality Assurance Manager
Hilton QA Manager auditing cleanliness, safety, and brand standards across 100+ hotels. Traveling onsite to evaluate properties and advise hotel leadership.
Account Investigator, Quality Assurance Refunds
Account Investigator analyzing patient accounts and resolving healthcare billing refunds for Zotec Partners. Investigating credit balances, refunds, and insurance adjustments remotely.
ICT Cabling Infrastructure Quality Assurance
ICT cabling quality assurance supporting SAIC’s U.S. Army Reserve network across 700+ locations. Reviewing construction submittals, advising on standards, and conducting infrastructure inspections.
Junior Penetration Tester
Junior Penetration Tester conducting web, network, and mobile penetration tests for SAIC’s state and local government customer. Documenting vulnerabilities and actionable remediation guidance.
Lead Tester
Lead Tester validating clinical databases, eCRFs, edit checks, and workflows for Medtronic’s global healthcare technology business. Supporting regulated clinical research testing and user acceptance activities remotely from India.
Senior Software Quality Assurance Engineer
Senior QA Engineer validating automated and manual tests for Parexel’s clinical e-Logistics applications. Improving software quality, test automation, and releases supporting clinical research.
Quality Assurance Engineer – Desktop Solutions
Quality Assurance Engineer testing CFRA’s desktop solutions and financial insights software. Automating web and AWS API tests, performance testing, and security validation.
Scientific QA – User Testing
Manual exploratory QA tester validating Luma, Dotmatics’ scientific research platform. Testing workflows, documenting defects, and improving usability across web applications.
QA – User Testing
Manual exploratory tester validating Luma scientific research software at Dotmatics. Testing scientific workflows, documenting defects, and improving SaaS product quality.
Director of Training – Quality Assurance
Director overseeing Ancora Education’s CDL training quality, compliance, and operational performance. Training leaders, analyzing testing results, and ensuring FMCSA and state regulatory adherence.
Lead Tester
Lead Tester validating clinical databases, eCRFs, edit checks, and integrations for Medtronic’s global healthcare technology research. Supporting user acceptance testing and regulatory compliance remotely from India.
Salesforce QA
Salesforce QA testing Sales and Service Cloud applications for a global automation and cloud technology provider. Managing automated tests and JIRA defects across international delivery teams.
QA Analyst, Credit Risk
QA Analyst reviewing mortgage files, fraud risks, and underwriting compliance for nesto, Canada’s digital mortgage lender. Supporting audits and credit-risk mitigation.
QA Analyst, Credit Risk
QA Analyst reviewing mortgage files, fraud indicators, and underwriting compliance for nesto, Canada’s digital mortgage lender. Supporting audits and credit-risk mitigation.
Associate QA Tester
Associate QA Tester validating cybersecurity products for GuidePoint Security, a cybersecurity solutions provider. Creating tests, tracking defects, and supporting automated QA for secure, high-performance applications.