Software Engineer, Agent Optimization – Evaluation
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Software Engineer building Reddit’s agent optimization and evaluation platform. Developing AI-agent benchmarking, orchestration, and feedback systems for software engineering workflows.
Responsibilities:
- Lead Reddit's agent optimization and evaluation platform
- Design and build an agent optimization harness for evaluating, benchmarking, and improving AI agents across software engineering workflows
- Develop Reddit-specific evaluation environments for measuring agent performance against meaningful engineering tasks
- Build systems for offline replay, automated evaluation, synthetic task generation, regression detection, and human-in-the-loop feedback
- Design model routing and orchestration strategies optimizing quality, latency, and cost
- Define metrics and experimentation frameworks measuring developer productivity and engineering velocity
- Build tooling to analyze agent behavior, investigate failures, and accelerate iteration
- Partner with Platform Engineering, Infrastructure, and Developer Productivity teams
- Shape Reddit's long-term strategy for a flexible, model-agnostic agent platform
Requirements:
- 5-12+ years of engineering experience
- Extensive experience building and shipping large-scale production software systems
- Experience designing evaluation, experimentation, benchmarking, or measurement systems for AI, search, developer platforms, or other complex distributed systems
- Strong understanding of modern LLMs, AI agents, and the engineering systems required to deploy them effectively
- Experience building systems that optimize across quality, latency, reliability, and cost
- Excellent software engineering fundamentals
- Ability to operate independently in ambiguous environments and drive technical direction for highly strategic initiatives
- Strong cross-functional collaboration skills and experience working across engineering, infrastructure, and platform organizations
Benefits:
- Comprehensive Healthcare Benefits and Income Replacement Programs
- 401k with Employer Match
- Global Benefit programs that fit your lifestyle, from workspace to professional development to caregiving support
- Family Planning Support
- Gender-Affirming Care
- Mental Health & Coaching Benefits
- Flexible Vacation & Paid Volunteer Time Off
- Generous Paid Parental Leave
- Equity in the form of restricted stock units
- Medical, dental, and vision insurance
- Flexible vacation and generous time off for vacation
- Parental leave
Similar Jobs
Senior Full Stack Developer
Senior Full Stack Developer modernizing school transportation platforms with C#, C++, React, and Claude Code. Building reliable core systems and cloud-native AI-enabled applications.
Full Stack AI/ML Developer – Digital Products
Full Stack AI/ML Developer building AI-enabled digital products for RTX, a global aerospace and defense company. Designing cloud-native data platforms, applications, APIs, and ML integrations.
Senior / Staff Software Engineer, Mobile
Senior software engineer building Clear Street Active, a high-performance trading platform for active traders. Developing responsive interfaces with real-time market data and low-latency execution workflows.
Full Stack Engineer
Senior Full Stack Engineer embedding AI-assisted coding, testing, and deployments for a global fintech trading company. Building Java tooling across Jira, GitLab, and CI/CD.
Java Software Developer
Java software developer delivering customer projects for Auralis Group, an IT innovation and venture studio. Building software, startups, and advanced technology solutions with career growth toward leadership.
Product Engineer
Product Engineer owning end-to-end features for Duvo’s AI operations platform. Shipping AI agents and enterprise workflow automation with customer impact.
Senior AI-Augmented Full-Stack Developer / Solutions Engineer
Senior AI-augmented full-stack developer building Nexus applications for an Australian IT hardware wholesaler. Integrating Ostendo ERP, LLMs, security, testing, and production systems.
Staff Engineer, Software
Staff Software Engineer building C#/.NET and React enterprise systems for Shield AI, a defense-tech company. Designing cloud integrations, event-driven services, and AI-powered business tools.
Security Engineering Team Lead
Security Engineering Team Lead leading Cyera’s security platforms, AWS architecture, detection automation, and agent fleet. Building secure AI trust infrastructure for enterprise customers.
Lead Software Engineer
Lead engineer embedded in Beacon Software’s acquired vertical SaaS companies. Shipping code, modernizing SDLCs, coaching teams, and deploying AI-powered product capabilities.
Lead Software Engineer
Lead Software Engineer embedded in Beacon’s acquired vertical SaaS companies. Shipping code, modernizing engineering practices, coaching teams, and deploying AI-powered product capabilities.
Oracle EBS Software Developer
Oracle EBS developer building PL/SQL financial systems for CGI Federal’s defense consulting programs. Supporting DLA and DAI cloud-hosted federal financial management, compliance, and analytics.
Director, Software Engineering
Director leading Databricks pipeline migration, AI-assisted engineering, and data mastering integration. Driving enterprise data-platform delivery for S&P Global’s global financial intelligence business.
Software Engineer – Medvitals
Software Engineer developing, testing, and supporting Medvitals application systems at Johns Hopkins School of Medicine. Translating complex clinical and business requirements into integrated software solutions.
Principal Engineer
Principal Engineer setting AWS architecture, service, and data direction for Yourgi’s AI-first pet-care marketplace. Leading platform modernization, reliability, and engineering standards across API, web, and mobile.
Senior Software Engineer
Senior Software Engineer building end-to-end React/TypeScript and C#/.NET features for InRule’s SaaS decision-intelligence platform. Delivering secure, maintainable software and leading complex engineering initiatives.
Senior Software Development Engineer, Full Stack
Senior full-stack engineer building Gifted Savings’ web platform for student investment portfolios and financial education. Leading responsive product development and technology decisions in a remote U.S. nonprofit team.
Software Development Engineer, Level 1
Software development engineer contributing to Dexcom’s continuous glucose monitoring and diabetes management technology. Coding, testing, deploying, and improving Dexcom Clarity software.
Software Development Engineer 1
Software development engineer contributing to Dexcom's continuous glucose monitoring software. Coding, testing, deploying, and collaborating on Dexcom Clarity to improve diabetes care.
Principal Software Engineer – Containers
Principal Software Engineer setting technical direction for Chainguard’s hardened container images, registry, and adoption tooling. Leading multi-team delivery for a company securing open-source software.