Member of Technical Staff, Frontier AI
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Frontier AI technical professional owning research evaluation, ML-oriented data design, and failure analysis. Improving deployed model and agent performance through evidence-based iteration.
Responsibilities:
- Own research and evaluation initiatives from problem framing through data design, quality calibration, and signal validation
- Define rigorous approaches for determining whether experimental results provide reliable and defensible research signal
- Analyse model and system failures to identify root causes, edge cases, and opportunities for improvement
- Evaluate datasets, experiments, and conclusions against appropriate quality thresholds
- Act as a quality gate when signal strength, data integrity, or supporting evidence is insufficient
- Design ML-oriented data systems, including task definitions, annotation schemas, rubrics, incentives, and supporting pipelines
- Structure data and evaluation workflows around downstream model-performance objectives
- Translate ambiguous real-world behaviour into measurable evaluation frameworks and new data categories
- Identify evaluation or dataset coverage gaps and recommend additional investment or iteration
- Develop quality-assurance processes that maintain consistent research standards
- Investigate model and system behaviour to identify recurring weaknesses and performance limitations
- Iterate on evaluations, datasets, feedback loops, and quality standards
- Use experimental findings to guide improvements in model or agent performance
- Determine when research directions should be expanded, revised, paused, or discontinued based on evidence
- Maintain a systems-level perspective focused on end-to-end AI performance
- Collaborate with researchers, domain experts, operators, and cross-functional teams during project kickoff, calibration, and iteration
- Communicate research findings, trade-offs, limitations, and signal strength to technical and non-technical stakeholders
- Translate research progress into evidence-grounded narratives
- Support alignment between experimental work and real-world system requirements
- Contribute technical judgement in ambiguous, high-impact research environments
Requirements:
- Experienced technical professional
- Strong professional judgement regarding research signal quality
- Experience designing ML-oriented datasets, evaluation frameworks, annotation systems, rubrics, or QA processes
- Ability to translate complex and ambiguous real-world system behaviour into structured research and evaluation opportunities
- Strong ownership mindset and comfort making decisions in uncertain or rapidly evolving environments
- Excellent written and verbal communication skills
- Ability to explain technical trade-offs, limitations, evidence quality, and research findings clearly
- Proven experience working directly with researchers, technical experts, or domain specialists during project calibration and iteration
- Systems-level understanding of model, agent, or AI-system performance
- Experience with reinforcement-learning environments, simulators, or feedback-driven training systems is advantageous
- Experience improving agentic systems or AI systems operating within real-world workflows is beneficial
- Prior work within applied research or production environments with direct impact on deployed systems is advantageous
- Experience designing evaluations for complex or real-world tasks is strongly valued
- Familiarity with expert incentive design or high-stakes technical research programmes is beneficial
- Work must be completed without using confidential or proprietary information belonging to any employer, client, institution, or other third party
Benefits:
- Fully remote work
- Full-time engagement
- Remote consulting opportunity
- Compensation of $600,000–$2,000,000/year
Similar Jobs
Member of Technical Staff, Enterprise AI
Enterprise AI researcher improving machine-learning evaluation, datasets, and agentic workflows for 24-MAG. Analyzing failure modes and experimental results across enterprise AI systems.
Member of Technical Staff, Coding Research
Coding research specialist evaluating frontier AI agents through benchmarks, datasets, and failure analysis. Building research tooling and methodologies with software engineers and AI researchers.
Software Engineer
Software Engineer developing full-stack applications and migrating data to AWS for First Advantage’s fulfillment team. Automating workflows, resolving technical issues, and mentoring engineers.
Landscape Developer
Landscape Developer advancing Commonland’s nonprofit landscape-restoration network across the UK and Ireland. Building partnerships, fundraising strategies, and nature-finance mechanisms for regenerative change.
Engineering Intern
Engineering Intern creating 2D and 3D CAD drawings for Custom Truck One Source’s specialized truck and equipment solutions. Maintaining revisions and resolving drawing errors with contractors.
z/OS System Programmer
z/OS System Programmer maintaining and modernizing mission-critical mainframe infrastructure for DXC Technology. Supporting z/OS operations, LPAR projects, security, performance, and distributed EU teams.
Senior Associate – Platform Development Engineering
Senior Associate developing scalable nCino lending and CRM platforms for Atlantic Union Bank. Defining architecture, roadmaps, requirements, and platform support.
Senior Associate – Platform Development Engineering
Senior Associate developing scalable nCino, Salesforce, and lending/CRM platforms for Atlantic Union Bank. Defining architecture, roadmaps, requirements, and technical solutions.
Protection & Control Engineering Associate II
Protection and Control Engineering Associate designing substation drawings and connection diagrams for Mesa Associates’ power delivery projects. Using AutoCAD for substation engineering design.
AI/ML Document Processing Developer
AI/ML developer building AWS Bedrock pipelines to classify and extract healthcare document data. Operating scalable Python backends, OCR, and structured-data workflows for The ACI Group.
Medior 3D XR Developer
Medior 3D XR Developer building Unity-based medical simulations for SyncVR Medical’s hospital XR platform. Creating AI-enhanced training modules with C#, xAPI, and Quest 3 optimization.
Senior Dynamics MPOS Developer
Senior D365 Commerce developer customizing MPOS and Store Commerce solutions for Pandora’s retail technology ecosystem. Supporting integrations, deployments, production issues, testing, and scalable releases.
Odoo Developer
Odoo Developer customizing Python, JavaScript, and PostgreSQL-based ERP applications. Building integrated Odoo solutions for global clients through a Philippine remote staffing firm.
Software Engineer
Software Engineer building React/TypeScript tools that help multilingual learners access rigorous classroom content. Extending Medley Learning’s Chrome extension, web apps, and AI-powered instructional supports.
Developer Research Coordinator – Contract
Developer Research Coordinator supporting MLH’s developer research through participant panels, qualitative analysis, and census studies. Fully remote contract role serving technology companies and developer communities.
C# Developer
C# Developer building scalable .NET backend services, APIs, and automation tools. Supporting FyerX’s internal infrastructure and diverse client technology platforms.
Senior Power Apps Developer – Power Platform, Microsoft 365
Senior Power Apps Developer building Power Platform applications, automation, and integrations for Booker DiMaio’s IRS Microsoft 365 contract. Supporting secure federal business solutions.
Software Engineer
Part-time Software Engineer evaluating AI-generated technical responses and documentation for 24-MAG’s remote consulting platform. Scoring accuracy, clarity, and usefulness with detailed written rationales.
Member of Technical Staff, Finance Research
Finance AI researcher developing benchmarks and evaluation frameworks for advanced financial reasoning and agentic systems. Conducting applied research across AI models, enterprise workflows, and quantitative finance.
Software Engineer
Contract Software Engineer contributing code, debugging, optimization, and documentation across major programming languages. Reviewing technical submissions to improve AI training data quality.