Machine Learning Engineer – Model Evaluation, Experimentation
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Machine learning engineer authoring experiments and evaluating frontier AI models for a leading AI lab. Implementing ML and reinforcement-learning tasks, analyzing results, and identifying model limitations remotely in the United States.
Responsibilities:
- Design well-defined, multi-step machine-learning tasks from real ML research ideas
- Implement changes, run training experiments, and analyze results to establish correct solutions
- Build tasks around reinforcement-learning concepts such as reward functions and training behavior
- Evaluate frontier models and identify where and why they fall short
- Compare notes with researchers and fellow experts to maintain consistency, rigor, and fairness
- Work in a tight feedback loop with the lab’s researchers
Requirements:
- MSc or PhD in machine learning, computer science, or another STEM field, or equivalent practical experience in a research-heavy domain
- 1+ years of experience in a research or research-engineering role
- Hands-on experience training and evaluating ML models and running experiments end-to-end, including setup, execution, and analysis
- Strong familiarity with large language models, including their capabilities, limitations, and evaluation techniques
- Working proficiency in Python and Git
- Comfort working in scripting and notebook environments
- Basic understanding of reinforcement learning, including reward functions and policy training, preferred
- Past experience in AI training, model evaluation, or benchmark/task authoring preferred
- High attention to detail and creativity in task design
- Strong written communication skills
- Ability to work independently through ambiguous, open-ended problems
- Ability to engage reliably for approximately 35 hours per week
Benefits:
- W-2 employment
- Payroll, benefits, and compliance administered by Cincinnatus LLC
- Fully remote work within the United States
- Approximately 35 hours per week
- Opportunity to be placed at a leading AI lab as part of its extended workforce
- Reasonable accommodations for qualified individuals with disabilities and disabled veterans throughout the job application process
Similar Jobs
Staff Machine Learning Engineer – Gen AI, Voice & Speech
Staff GenAI Engineer building scalable voice and speech AI platforms for Weave, a healthcare communications technology company. Leading ML infrastructure, architecture, and cross-team technical strategy.
Staff ML Engineer – ML/AI
Staff ML/AI Engineer architecting Lyra Health’s enterprise AI platforms for evidence-based mental health care. Driving generative AI infrastructure, reliability, security, and clinical precision.
Senior ML Engineer – MLOps
Senior MLOps Engineer building production ML and generative AI infrastructure for Lyra Health, an evidence-based mental healthcare provider. Developing platforms, evaluation frameworks, and AI product services.
Applied AI/ML Engineer
Applied AI/ML Engineer building NLP, LLM, and knowledge-graph systems for Intangible’s interactive 3D creative tools. Developing AI assistants that help users visualize and communicate spatial ideas.
Senior Machine Learning Engineer
Senior ML Engineer developing production computer-vision models for GBG’s digital identity verification technology. Deploying, evaluating, and improving AI systems while mentoring CVML engineers in an Agile environment.
Staff ML Engineer, GenAI, Voice & Speech
Staff ML Engineer building scalable voice and speech GenAI infrastructure for Weave’s AI-powered products. Designing models, platforms, and distributed services for high-volume customer experiences.
Senior Machine Learning Software Engineer
Machine Learning Senior Software Engineer building model validation, optimization, and safety systems for Akamai’s distributed cloud platform. Developing secure inference pipelines, guardrails, and model evaluation frameworks.
Forward Deployed Machine Learning Engineer
Forward Deployed ML Engineer building benchmarks, model evaluations, and backend infrastructure for Protege’s secure AI training-data exchange platform. Partnering with researchers, customers, and DataLab to turn evaluation engagements into products.
Machine Learning Scientist 5 – Forecasting Aggregation
Machine Learning Scientist building Netflix’s ad-campaign forecasting models. Developing interpretable predictions for inventory, delivery risk, reach, and frequency at scale.
ML Software Engineer
ML Software Engineer building multimodal data pipelines, foundation models, and inference systems. Advancing reliable autonomous freight vehicles through physical AI and production-grade tooling.
Staff ML Engineer – Scene Generation
Staff ML Engineer leading neural rendering and generative modeling for Torc Robotics’ autonomous truck software. Building scalable Camera, LiDAR, and Radar simulation pipelines for AV training and validation.
VP of Research, Machine Learning
Leading research and production ML systems for A1’s proactive smart assistant. Defining reasoning, evaluation, safety, and orchestration for reliable everyday task completion.
Machine Learning Engineer, Speech – Joint Audio-Video Modeling
Research/ML Engineer specializing in audio generation and AI systems at Cantina. Leading development of advanced speech and audio models for joint audio-video applications.
Machine Learning Engineer
Machine Learning Engineer at Interval solving technical problems related to AI, privacy, and distributed systems. Collaborating to build models using enterprise data while maintaining privacy.
Machine Learning Engineer, Especialista II – Dados, Consumidor
Machine Learning Engineer responsible for developing ML solutions and pipelines at Grupo Boticário. Collaborating in a dynamic data ecosystem for beauty brands globally.
Senior Machine Learning Engineer
Senior Machine Learning Engineer developing core AI systems for proactive smart assistant solutions. Collaborating with cross-functional teams to deliver impactful AI products at scale.
Senior Machine Learning Engineer
Senior Machine Learning Engineer at A1 Engineering building AI-native systems. Owning critical ML subsystems, collaborating with teams, and ensuring production reliability.
Senior Machine Learning Engineer
Senior Machine Learning Engineer building and shipping machine learning systems for proactive AI products. Collaborating with research teams and mentoring other engineers.
Senior Machine Learning Engineer
Senior Machine Learning Engineer building and shipping ML systems that are reliable in production. Collaborating with cross-functional teams and mentoring ML engineers to enhance performance.
Member of Technical Staff, Machine Learning
Join A1 Engineering as a Member of Technical Staff, Machine Learning. Build ML components and improve production systems with real-world applications.