Senior Software Engineer, Open Harness Engineering
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Senior software engineer building open-source AI agent harnesses and evaluation systems at NVIDIA. Improving agent reliability, performance, and safety through benchmarks, sandboxing, and trace analysis.
Responsibilities:
- Define and evolve the technical approach for evaluating and improving open agent harnesses across frameworks, models, and benchmark environments
- Build and operate scalable, reproducible evaluation systems spanning CI, scheduled compute, sandboxed execution, artifact provenance, trace capture, and analysis
- Diagnose agent failures from traces, logs, tool results, and benchmark artifacts
- Separate harness, evaluator, environment, inference, and model causes
- Design controlled experiments and promotion criteria to distinguish genuine harness improvements from noise, benchmark artifacts, or model-specific gains
- Ship harness and runtime improvements, including upstream open-source contributions, detectors, regression tests, and maintainable documentation
- Partner with Relay, Hermes, model, infrastructure, and research teams to turn recurring optimization needs into reusable runtime, trace, and evaluation capabilities
Requirements:
- Bachelor’s or Master’s degree in Computer Science, Artificial Intelligence, Applied Math, or a related field, or equivalent experience
- 8+ years of hands-on software-engineering experience
- Demonstrated technical ownership of production systems
- Architecture design leadership
- Expert Python skills
- Working proficiency in Rust, Go, C++, or TypeScript
- Ability to debug systems across language and process boundaries
- Experience building or extending LLM agents, coding agents, tool-use loops, model-provider integrations, developer tools, or evaluation infrastructure
- Understanding of asynchronous execution, subprocesses, containers, sandboxing, callbacks, retries, networking, and distributed compute
- Experience crafting reproducible experiments, benchmark methodology, performance investigations, regression gates, or reliability analysis under nondeterministic conditions
- Record of shipping and maintaining open-source or developer-facing software with sound testing, API development, documentation, and code reviews
Benefits:
- Highly competitive salaries
- Comprehensive benefits package
- Equity
Similar Jobs
Staff Engineer, ServiceNow – ITSM & Scripting
Staff Engineer developing ServiceNow ITSM applications, scripts, integrations, and workflows. Delivering scalable enterprise solutions for Nagarro’s digital product engineering business.
Senior Engineering Systems Specialist
Senior CAD systems specialist optimizing CATIA and engineering workflows for Elevate Aircraft Seating’s aircraft seat manufacturing. Governing standards, automation, user support, and digital engineering improvements.
Senior Engineering Systems Specialist
Senior Engineering Systems Specialist optimizing CATIA systems for Elevate Aircraft Seating’s innovative aircraft seat manufacturing. Governing CAD standards, automation, support, and engineering workflows.
Senior Software Engineer – Trust
Senior Software Engineer building backend services for Veriff’s AI-powered identity verification platform. Integrating authoritative data sources and strengthening global trust signals.
Senior Fullstack Engineer
Senior Fullstack Engineer developing and enhancing secure client-facing portals for regulated financial services. Building React/NestJS features end-to-end and supporting production reliability.
Senior Full-Stack Software Engineer
Senior full-stack engineer building React, Node.js, and Go features for Ontrac Solutions customer engagements. Refactoring legacy codebases, securing services, and designing PostgreSQL and MongoDB data systems.
Senior Staff Engineer – Logging, Chronosphere
Sr. Staff Engineer building Chronosphere’s scalable telemetry collection and ingestion platform. Developing Go-based distributed systems for Palo Alto Networks’ observability product.
Senior Full Stack Engineer
Senior Full Stack Engineer building React and TypeScript operator workflows for SimSpace’s cyber simulation platform. Contributing to backend integrations, platform observability, and resilient asynchronous operations.
Senior Full Stack Developer – C#, Angular
Desenvolvedor Full Stack construindo APIs .NET e interfaces Angular para a Alterdata Software. Projetando soluções web escaláveis para segmentos contábil, empresarial e varejo.
Senior Software Development Engineer – LLMs, GenAI
Senior engineer building scalable LLM, GenAI, and AI-agent platforms for CVS Health. Designing secure, observable, production-ready healthcare technology solutions.
Senior Full Stack Engineer – .NET/React
Senior Full Stack Engineer building .NET/React applications for a media-technology company powering advertising workflows. Developing cloud-based services, front-end components, and CI/CD pipelines.
Senior Full Stack Engineer – .NET/React
Senior Full Stack Engineer building scalable .NET/React applications for a media-technology advertising platform. Developing backend services, front-end components, cloud infrastructure, and CI/CD pipelines.
Senior Software Developer, Payroll-Deducted Loans Squad
Desenvolvedor backend sênior criando soluções escaláveis para a plataforma bancária da Fineasy Tech. Liderando arquitetura, qualidade técnica, documentação e resolução de incidentes de produção.
Mid-Level Software Developer, Payroll Loans Squad
Desenvolvedor backend para soluções financeiras escaláveis da Fineasy Tech, parceira tecnológica do Banco Topázio. Atuando com microsserviços, APIs RESTful e sistemas distribuídos.
AI Software Architect – MCP, JS
Software Architect building scalable MCP, Node.js, and AI orchestration platforms. Establishing enterprise architecture standards and reusable integrations for a European FinTech company.
Fullstack Engineer, Developer
Fullstack Engineer building secure, scalable Azure applications for The Whiteam’s international technology consulting clients. Managing Kubernetes, Terraform, CI/CD, and platform operations remotely from the EU.
Tech Lead
Full-stack Node.js and React engineer building cloud-native applications for Charles River, a contract research organization supporting pharmaceutical and biotech therapies. Designing scalable APIs, responsive UIs, and secure backend systems.
Tech Lead/Software Architect – React, Next.JS, Node.JS, AI, AWS
Tech Lead shaping cloud-native web and mobile platforms at Software Mind. Leading architecture, AWS serverless development, AI innovation, and engineering teams.
Senior Software Engineer – Formal Methods, Agentic Systems
Senior Software Engineer productionizing formal-methods and agentic AI systems. Building provably correct solutions for climate, energy, utilities, and infrastructure clients.
Senior Software Developer – Tech Lead
Senior Java Tech Lead building distributed middleware and backend systems for Kyivstar.Tech, a Ukrainian telecom technology company. Designing microservices, APIs, and high-performance services.