Staff AI Engineer
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Staff AI Engineer designing reliable, secure and cost-efficient production agents for Cobalt’s pentesting platform. Setting agent architecture, evaluation, observability and engineering standards across teams.
Responsibilities:
- Set technical direction for agent systems across teams, including orchestration, tool use, context and memory management, human oversight and failure recovery
- Design, build and operate shared foundations for evaluations, observability, tracing, execution controls and model access with routing and failover
- Establish evaluation practices for task completion, accuracy, false positives, security and cost per successful task
- Build representative test cases with domain experts and turn production failures into regression tests
- Improve resilience to tool failures, timeouts, rate limits and partial execution with limits on retries, runtime and spend
- Build scoped permissions, customer data isolation, audit trails and safe tool/code execution with security and platform teams
- Establish engineering standards for CI evaluations, prompt/model versioning, regression testing, cost and latency budgets, and staged rollouts
- Lead model selection and cost optimization by benchmarking models against real workloads
- Partner with teams on production readiness, incident investigation and agent reliability
- Help engineering teams use AI coding agents across planning, implementation, review and testing
- Mentor engineers and technical leads through design reviews, pairing, internal talks and written guidance
- Create a technical roadmap with product, research and security leaders
Requirements:
- Significant software engineering experience, typically 8–10+ years, with a track record of technical leadership across teams
- Hands-on experience building and operating LLM-based agents in production
- Track record of improving agent reliability through evaluations, non-deterministic failure investigation and measurable success-rate improvements
- Strong knowledge of agent architectures, including tool use, orchestration, planning, context and memory management
- Experience running LLM workloads through managed cloud model platforms or direct provider APIs, including rate limits, failure handling and cost tracking
- Strong backend and distributed systems fundamentals, including APIs, asynchronous workflows, concurrency and fault tolerance
- Experience with production observability, incident investigation and safe rollout of system changes
- Understanding of security risks in agent systems, including prompt injection, unsafe tool execution and sensitive-data exposure
- Evidence of impact beyond individual code, such as shared platforms, improved standards or mentoring engineers
- Experience with agent orchestration, tool calling and structured outputs
- Experience with Model Context Protocol (MCP), APIs and permission-controlled tool execution
- Experience with automated evaluation harnesses, domain-specific test datasets and regression testing in CI
- Experience with distributed tracing, structured logs and metrics for multi-step agent workflows
- Experience with managed cloud model platforms, direct provider APIs and multi-provider gateways
Benefits:
- Competitive compensation and an attractive equity plan
- 401(k) program (US) or pension (EU)
- Medical, dental, vision and life insurance (US) or statutory healthcare (EU)
- Wellness stipend
- Work-from-home equipment and Wi-Fi stipend
- Learning and development stipend
- Flexible, generous paid time off
- Paid parental leave
- Ongoing mentorship opportunities
Similar Jobs
AI Engineer – Freelance
AI Engineer building generative AI, LLM, and RAG solutions for HRM clients. Designing scalable platforms and supporting software delivery from analysis through deployment.
Staff Software Engineer – AI
Staff Software Engineer building and deploying scalable AI/ML features for 8am’s SaaS platform. Developing Python, LLM, data infrastructure, and cloud deployment solutions for professional services.
Senior C# Developer, AI Experience
Remote senior C#/.NET developer building APIs and analyzing an enterprise data platform. Later-phase work includes LLM integration, RAG, and GenAI solutions.
Software Engineer – AI Systems, Go
Software Engineer building Go-based production AI systems and durable agent workflows for Stanbridge University. Designing reliable, observable, multi-provider AI infrastructure and backend services.
Working Student – AI Engineering
Working Student building Python AI applications, RAG workflows, and agent systems at Hamburg-based data and AI engineering company AgentixLake. Supporting evaluation, testing, deployment, and monitoring.
Staff Applied AI Engineer
Staff Applied AI Engineer owning AI portfolios and shipping agentic applications. Modernizing collectibles authentication, grading, vaulting, and trading through applied AI.
Staff Software Engineer – AI Platform
Staff Software Engineer building Aledade’s shared AI infrastructure for secure foundation-model access. Developing scalable model gateways, cloud integrations, observability, and self-service platform capabilities.
Divisional Artificial Intelligence Engineer III
AI platform engineer leading architecture, reliability, and adoption for shared LLM infrastructure. Building reusable orchestration, observability, and production services across products.
Forward Deployed AI Engineer
Forward Deployed AI Engineer deploying production AI solutions for Keyrus, an enterprise intelligence consultancy. Integrating APIs, cloud platforms, enterprise systems, and client workflows.
Staff Applied AI Engineer
Staff Applied AI Engineer owning business-domain AI portfolios at Collectors, a collectibles authentication, grading, and trading technology company. Building production agents and reusable application patterns.
AI Architect, Software Architect
Arquitecto de IA Generativa y software para knowmad mood, compañía de transformación digital. Diseñando arquitecturas, soluciones empresariales y servicios de IA para clientes.
Java Software Architect, AI
Arquitecto Java y especialista en IA Generativa para knowmad mood, compañía de transformación digital. Diseñando arquitecturas, modernizando aplicaciones y asesorando a clientes en soluciones de IA.
Senior Libraries Engineer – AI, HPC
Senior Libraries Engineer developing GPU- and CPU-accelerated libraries for AI, image processing, and scientific computing at NVIDIA. Optimizing algorithms and leading library engineering projects.
AI Engineer, Forward Deployed
AI Engineer building production Claude agents, RAG pipelines, and MCP integrations for Collectiv’s enterprise analytics and AI consulting clients. Consulting directly with stakeholders from a fully remote U.S.-based role.
AI Engineer, Forward Deployed – Copilot, Azure AI Foundry
AI Engineer building production Microsoft Copilot and Azure AI Foundry solutions for Collectiv’s enterprise consulting clients. Developing RAG pipelines, agents, integrations, evaluations, and production-ready governance.
Engineer, Software Architect, AI Practitioner
Software-Architekt entwickelt moderne TypeScript-Enterprise-Anwendungen für DAI Software. Verantwortlich für Architektur, Entwicklung, Testing und Deployment KI-gestützter Lösungen.
AI Engineer, Analyst III
Pessoa AI Engineer no Grupo Boticário, líder brasileiro de beleza, desenvolvendo integrações com LLMs para o OMS. Automatizando fluxos, reduzindo trabalho manual e melhorando a experiência do usuário.
AI Engineer – New College Graduate
New graduate AI engineers building Bitdeer’s full-stack AI cloud infrastructure. Developing cloud, Kubernetes, model-serving, and AI Agent platforms for globally distributed workloads.
Junior AI Engineer
Engenheiro de IA júnior desenvolvendo chatbots, copilots e agentes autônomos baseados em LLMs. Integrando APIs, pipelines RAG e sistemas de observabilidade em produção.
AI Architect
AI Architect designing secure, scalable AI and ML solutions for a federal healthcare modernization program. Leading enterprise architecture, cloud integration, MLOps, and responsible AI practices.