Software Engineer – AI Evaluation
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Software Engineer building AI evaluation systems for Board Intelligence’s board-governance platform. Automating quality measurement across full-stack tools, data pipelines and AI products.
Responsibilities:
- Design and build the AI evaluation suite used to measure and improve AI output quality
- Build across the full stack, including data pipelines, backend services and front-end tools
- Work with prompt engineers and subject matter experts to turn knowledge into tests, metrics and tooling
- Help define AI quality measurement practices
- Automate evaluation processes and results within the software development lifecycle
- Contribute over time to AI-powered products
- Deliver an evaluation suite used by engineering teams after three months
- Extend evaluations across all AI products and integrate them into the SDLC after six months
- Establish evaluation as a standard part of AI delivery and become a trusted voice on AI quality after twelve months
Requirements:
- Upper mid-level to senior professional software engineering experience
- Experience building AI powered products or tooling
- Full-stack experience across front end and back end
- Comfortable working independently and in ambiguity, setting own direction
- Experience collaborating with non-engineers and turning their expertise into working software
- Strong focus on quality, testing and attention to detail
- Experience using AI coding tools in day-to-day work
- Fluent written and spoken English
- Nice to have: experience building evaluation suites for AI products
- Nice to have: experience with Python and TypeScript
- Nice to have: experience building internal tools, testing frameworks or data pipelines
- Nice to have: experience at a startup or in a small team building things end to end
Benefits:
- Pension scheme
- Personal performance bonus
- 26 days holiday each calendar year
- Bupa health & dental cover
- Group life assurance
- EAP
- Cycle to work scheme
- Diversity, equity and inclusion
- Reasonable adjustments accommodated during the hiring process
Similar Jobs
Senior AI Engineer
Senior AI Engineer building and operating production LLM workflows for Jeeves' stablecoin-native banking platform. Owning AI systems end to end across design, deployment, evaluation, and monitoring.
Senior Full Stack Software Engineer, AI-Driven Platform
Senior Full Stack Engineer building scalable AI biomanufacturing software for pharmaceutical and biotech companies. Developing APIs, interfaces, and platform features that improve manufacturing efficiency and reduce waste.
Senior Customer AI Engineer
Senior Customer AI Engineer guiding customers from proof-of-concept to production on Nebius’s AI cloud platform. Optimizing reliable, scalable, and cost-efficient AI workloads.
Senior AI Engineer, AI FDE
Senior AI Engineer building and deploying production GenAI applications for Databricks customers. Advising customers and shaping AI product roadmaps through Databricks professional services.
Machine Learning/AI Engineer
Senior AI Engineer building production GenAI and agentic workflows for Ambush’s financial-services clients. Developing Python/FastAPI, SQL, RAG, and cloud-based AI systems.
Senior iOS Engineer, AI First
Senior iOS Engineer building AI-assisted seller and buyer apps for OPENLANE’s digital used-vehicle marketplace. Leading SwiftUI/UIKit architecture, testing, releases, and cross-platform mobile collaboration.
Senior Software Engineer – AI-Augmented Development
Senior Software Engineer building AI-augmented hyperautomation and workflow-management software at German IT provider DATAGROUP. Developing secure fullstack applications, APIs, CI/CD pipelines, and test strategies.
AI Engineer
AI Engineer owning production LLM workflows for Jeeves’ stablecoin-native banking platform. Building reliable AI systems for finance automation, monitoring, and regulated financial decisions.
AI Engineer
AI Engineer building end-to-end LLM workflows for Jeeves’ stablecoin-native banking platform. Shipping reliable finance automation with evaluation, monitoring, and backend integration.
AI Engineer
AI Engineer owning production LLM workflows for Jeeves’s stablecoin-native banking platform. Building reliable, evaluated AI systems for enterprise finance operations.
AI Engineer
AI Engineer owning production LLM workflows for Jeeves’ stablecoin-native banking platform. Building, evaluating, and monitoring AI systems for finance automation.
AI Developer
AI builders developing real projects in recorded OpenCode sessions for Mercor’s AI research partner. Using provided open-weight and Gemini models on flexible, remote tasks.
Builder, Developer – AI Coding Sessions
AI builders develop real projects in recorded OpenCode sessions using provided open-weight and Gemini models. Mercor supports AI research into how people build with coding assistants.
Director, AI Engineering
Director leading responsible AI engineering, foundation models, and scalable infrastructure at Capital One. Delivering governed AI products that transform customer banking experiences.
Product Engineer – AI
Product Engineer building full-stack AI-powered products for Board Intelligence, a technology and advisory firm improving board effectiveness. Owning enterprise solutions from client needs through scalable system design and delivery.
AI Forward Deployed Engineer – AI FDE
AI engineer deploying multi-agent workflows for insurance customers through Shift Technology’s SaaS platform. Integrating, productionizing, and monitoring AI agents across claims systems.
AI Engineer – Mid-Level
Engenheiro de IA desenvolvendo chatbots, copilots e agentes autônomos com LLMs. Projetando pipelines RAG e sistemas seguros, observáveis e performáticos em produção.
Developer – AI Developer, Temporary
AI Developer using Claude Code to develop, test, and optimize generative AI software solutions. Temporary remote project with possible continuation at Azapi Solutions.
Applied AI Engineer
Applied AI Engineer building reliable agentic AI for R1’s healthcare revenue-cycle workflows. Designing, evaluating, and deploying production systems with clinical and engineering teams.
AI Engineer
AI Engineer building production AI/ML and LLM systems for Robots and Pencils’ enterprise clients. Developing pipelines, agentic features, RAG workflows, and model evaluations.