AI Interaction Evaluator – Codex, Claude Code
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Senior engineers evaluating AI coding-agent interactions for G2i. Assessing reasoning, explanations, outputs, and engineering judgment across Codex, Claude Code, and Cursor.
Responsibilities:
- Evaluate AI-generated coding interactions end-to-end
- Judge whether outputs are useful, correct at a high level, and aligned with how a strong engineer would think
- Assess the quality of explanations and reasoning, not just code
- Distinguish between different levels of response quality
- Provide clear, opinionated feedback on what worked, what did not, and what felt off or misleading
- Help define what great looks like when interacting with tools like Cursor
- Evaluate whether AI coding agents' responses, preambles, reasoning, and outputs demonstrate strong engineering judgment
- Participate in a take-home evaluation exercise and one behavioral interview
Requirements:
- Highly experienced software engineer at Senior+ level
- Staff- or Principal-level engineer, or equivalent experience
- Strong background in TypeScript/JavaScript or Python
- Hands-on experience using OpenAI Codex, Claude Code, and Cursor
- Deep familiarity with modern AI-assisted development workflows
- Ability to evaluate code without fully executing or deeply reviewing every line
- Comfortable giving direct, opinionated feedback
- High standards for engineering quality
- Comfortable making subjective but rigorous judgments
Benefits:
- Possible extension beyond early May
- Flexible part-time engagement of approximately 10–20 hours per week
Similar Jobs
French-Speaking Mac Users – Application-Level AI Audio Testing
French-speaking Mac freelancers testing enterprise AI audio applications. Evaluating generated audio, application behavior, bugs, and instruction adherence.
Senior AI Analyst
Senior AI Analyst automating marketing data workflows for Improvado's unified marketing intelligence platform. Delivering SQL DataMarts, pipelines, and AI-driven analytics solutions for enterprise clients.
Business Consultant — AI, Data & Digital Transformation
Consultant IA et Data accompagnant les clients de Konecta, fournisseur mondial de gestion client et d’externalisation digitale. Cadrage des usages, feuilles de route et déploiement de transformations numériques à impact mesurable.
AI Safety Red Teamer
AI Safety Red Teamer stress-testing frontier AI models for Mercor’s AI research partners. Designing adversarial prompts, finding vulnerabilities, and improving model safety and alignment.
AI Safety Practitioner
AI Safety Practitioner evaluating frontier AI models for Mercor, which partners with AI labs and enterprises to train advanced systems. Assessing sensitive content, applying safety rubrics, and providing structured alignment feedback.
Digital Transformation, AI Management
AI transformation specialist leading automation, Microsoft 365 adoption, and process improvement. Supporting an Australian automotive aftermarket company through roadmap, governance, training, and vendor management.
AI Model Training Expert – Face Motion Data Contributor
Face-motion contributor recording short videos for Perle AI's computer vision training data. Completing simple camera-based tasks remotely across six eligible countries.
AI Model Training Expert – Face Motion Data Contributor
Face-motion contributor recording short videos for Perle AI’s computer vision training data. Completing simple camera-based tasks for future paid AI training projects.
AI Workforce Enablement Lead
AI workforce enablement lead driving adoption and behavior change for Progressive Leasing’s FinTech transformation. Designing manager toolkits, learning programs, and AI-enabled workflow support.
Principal Technologist – AI and Data Center Networking
Principal Technologist architecting AI/HPC networking and data center solutions for DriveNets, a high-scale networking software company. Leading strategic pre-sales engagements, proofs of concept, and technical go-to-market enablement.
AI Creator
AI Creator producing performance video, static, and audio creatives for HOLYWATER TECH’s AI-driven entertainment products. Developing conversion-focused concepts for Meta, TikTok, and Snapchat.
AI Change Lead
AI Change Lead driving adoption, learning, communications, and governance for D-ploy’s enterprise AI initiatives. Coordinating stakeholders, events, AI Champions, rollout plans, and programme reporting.
Technical AI Quality Coordinator
Technical AI Quality Coordinator improving AI data quality for RWS’s global content and language services. Automating checks, building testing frameworks, and monitoring annotation and model-evaluation pipelines.
Director of AI and Educational Technology
Teach For All leader advancing global teacher-workforce policy and government partnerships. Building coalitions and supporting network partners across eligible remote locations.
Working Student AI Transformation
Working student helping casavi, a German PropTech company, turn AI ideas into practical tools and workflows. Supporting internal AI adoption across property-management software teams.
AI Safety Experts – English, Malay
AI red teaming expert probing frontier models for Mercor, which partners with AI labs and enterprises to train advanced models. Generating reproducible attack data and vulnerability reports to improve AI safety and robustness.
AI Safety Expert, English, Danish
AI red team expert attacking conversational models for Mercor, which partners with AI labs and enterprises to train frontier systems. Generating reproducible vulnerability data, attack cases, and safety evaluations for customers.
Workflow & AI Agent Engineer
Workflow and AI Agent Engineer supporting Chatarmin’s B2B e-commerce SaaS clients. Owning onboarding, support, API/LLM solutions, and product feedback loops.
AI Strategy – Digital Implementation
Werkstudent:in entwickelt Website, App und Klinik-Dashboard für Xeno Patchs wachsendes MedTech-Produktportfolio. Integration digitaler Bausteine, Feature-Tests und schnelle KI-gestützte Umsetzung neuer Funktionen.
Senior Analyst, Artificial Intelligence
Analista Sênior criando soluções de IA, LLMs e pipelines de dados para a digitalização industrial da Vetta. Projetando arquiteturas escaláveis em Databricks, AWS e Azure para otimizar operações e apoiar a descarbonização.