Senior Python Engineer – AI Code Evaluation
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Senior AI interaction evaluator assessing Codex, Claude Code, and Cursor outputs for engineering judgment. Providing rigorous feedback on reasoning, explanations, and developer trust.
Responsibilities:
- Evaluate AI-generated coding interactions end-to-end
- Judge whether outputs are useful, correct at a high level, and aligned with strong engineering judgment
- Assess the quality of explanations and reasoning, not just code
- Distinguish between different levels of response quality
- Provide clear, opinionated feedback on what worked, what did not, and what felt off or misleading
- Help define what great looks like when interacting with tools such as Cursor
- Make subjective but rigorous judgments about AI coding-agent interactions
Requirements:
- Highly experienced software engineer at Staff / Principal level or equivalent experience
- Strong background in TypeScript / JavaScript or Python
- Hands-on experience using OpenAI Codex, Claude Code, and Cursor
- Deep familiarity with modern AI-assisted development workflows
- Ability to evaluate code without fully executing or deeply reviewing every line
- Comfortable giving direct, opinionated feedback
- High standards for engineering quality
- Nice to have: experience with Cursor or similar AI-first IDEs
- Nice to have: prior exposure to prompt design or evaluation workflows
- Nice to have: experience mentoring senior engineers or defining engineering standards
- Must complete a take-home evaluation exercise and one behavioral interview
- Must agree to a simple background check for the project
Benefits:
- $100–$200/hour rate
- Approximately 10–20 hours/week
- Contract through early May, with possible extension
- Start ASAP
Similar Jobs
Back-end Developer
Back-end Developer building .NET Core microservices for property-management technology. Supporting real-time collaboration and scalable cloud products for Australian real estate agencies.
Senior Go Engineer – AI Code Evaluation, Codex, Claude Code
Senior engineer evaluating OpenAI Codex, Claude Code, and Cursor interactions for G2i. Assessing AI-generated reasoning, code quality, and engineering judgment.
Backend Developer – Senior
Desenvolvedor Backend Kotlin sênior atuando para banco digital brasileiro. Construção de soluções com Spring, bancos de dados, AWS e testes unitários.
Java Developer, Mid-Level
Java Developer building Java 17 and Spring Boot backends for CI&T’s global B2B data integration platform. Improving performance, scalability, reliability, and data integrity in a collaborative multicultural team.
Principal Python Engineer – FastAPI, Azure, AKS
Principal Python Engineer building FastAPI APIs, Azure infrastructure, and scalable data solutions for HumanIT in Portugal. Deploying with Terraform, AKS, Kubernetes, and Docker.
Senior Software Engineer – Backend Focused
Senior Full Stack Engineer building scalable digital services for U.S. government agencies. Developing cloud-based applications, APIs, and integrations while mentoring engineers and improving delivery practices.
Desarrollador Java Backend – APIs, Microservicios
Desarrollador Java backend creando APIs y microservicios para IDS Comercial, consultora mexicana de TI. Integración de seguridad, REST, bases de datos y CI/CD en proyectos financieros.
Senior Backend Developer
Desenvolvedor Backend .NET Sênior criando APIs RESTful e microsserviços para a consultoria de dados Rox Partner. Liderando soluções escaláveis, cloud, mensageria, CI/CD e observabilidade.
Mulesoft API Developer
MuleSoft API Developer building enterprise APIs and integrations for Resilient Co. across CRM, ERP, SaaS, and internal systems. Designing secure, reusable Mule 4 solutions with API-led connectivity and CI/CD support.
Java Developer – Berlin, Remote
Java Developer creating software solutions for SEITENBAU’s German clients. Implementing user stories, automated tests, code reviews, and CI/CD pipelines in agile Scrum teams.
Senior .NET Legacy Modernisation
Senior .NET Developer modernising TXP’s enterprise applications across legacy and hybrid environments. Maintaining C#, WinForms and WebForms systems while integrating REST, SOAP and XML services.
Senior Backend Engineer, Golang
Senior Golang Backend Engineer scaling analytics systems for BlueThrone’s global mobile products. Building APIs, migrating Node.js services, and optimizing data, caching, messaging, and infrastructure for millions of users.
Software Engineer – Backend
Backend engineer building cloud tooling that accelerates customer implementations for VibeIQ’s digital product collaboration platform. Owning scalable infrastructure features from design through deployment and monitoring.
Principal Azure Data Platform Architect – Databricks, MLOps Lead
Azure Data Platform Architect building secure Databricks and MLOps infrastructure for Florida government oversight. Engineering multi-agency pipelines, monitoring, governance, and State ownership transition.
Software Engineer, Back-End – Mail
Back-end engineer building scalable AI orchestration services and real-time infrastructure for Superhuman’s productivity platform. Collaborating across product teams to deliver reliable AI-powered features.
Senior Python Developer
Senior Python Developer building private cloud solutions for global clients at Software Mind. Designing backend applications, migrating workloads, and supporting production platforms.
Technical Architect, ServiceNow
SAFE/ServiceNow Technical Architect designing enterprise ServiceNow solutions for CGI’s U.S. federal consulting business. Guiding Agile teams across architecture, integrations, automation, governance, and platform strategy.
Senior Python Engineer, Document App
Senior Python Engineer building real-time collaborative document editing systems for PandaDoc’s document workflow automation platform. Owning persistence, event-driven services, reliability, and AI-powered document workflows.
Senior Python Engineer, Document App
Senior Python Engineer for PandaDoc shaping the Document App Track with innovative solutions. Collaborate with a diverse team and tackle technical challenges in a fully remote environment.
Senior Python Engineer, Document App
Senior Python Engineer enhancing document management within the team at PandaDoc. Contributing to architecture and facing technical challenges in a fast-paced environment.