AI Research Engineer
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
AI Research Engineer building multilingual LLM evaluation pipelines and annotation automation. Bridging applied NLP research with production engineering at RWS, a language-services company.
Responsibilities:
- Design, build, and maintain automated execution pipelines for continuous LLM benchmarking across diverse languages, tasks, and models
- Develop and roll out LLM-assisted pre-annotation and quality assurance pipelines for production data annotation
- Implement and refine LLM-as-a-judge pipelines, custom evaluation metrics, and scoring algorithms
- Build experiment infrastructure, datasets, and execution pipelines for multilingual LLM research studies and thought leadership reports
- Translate applied NLP research into scalable prototypes and integrate research tools into production operations with the data engineering team
- Collaborate on research findings, benchmark reports, and thought leadership publications
- Prototype emerging AI capabilities including agentic AI evaluation, synthetic data quality certification, and domain-specific LLM assessment
- Maintain documentation, code bases, and technical guides for evaluation frameworks and tools
Requirements:
- Master’s degree (MSc or equivalent) in Computer Science, Artificial Intelligence, Computational Linguistics, or a related technical field
- 3–5+ years of hands-on experience in NLP or ML engineering
- Advanced proficiency in Python and deep learning/NLP frameworks, including PyTorch or TensorFlow and Hugging Face transformers
- Proven experience with LLM orchestration frameworks such as LangChain or LlamaIndex
- Experience with automated evaluation techniques such as LLM-as-a-judge and benchmarking harnesses
- Expertise in data manipulation, structured evaluation data processing, and analysis using pandas, NumPy, or related tools
- Practical experience deploying and managing ML workflows on Azure, AWS, or Vertex AI
- Experience building scalable data or experiment orchestration pipelines
- Familiarity with multilingual NLP, machine translation, or computational linguistics
- Knowledge of localization workflows, annotation platforms, or human-in-the-loop data pipelines
- Track record of contributing to open-source NLP projects, evaluation frameworks, or published technical benchmarks
Benefits:
- Equal opportunity employment and a work environment free of discrimination and harassment
- Opportunities to grow as an individual and excel in their career
- Inclusive workplace valuing diversity
Similar Jobs
AI Research Engineer – Agentic Reinforcement Learning
AI Research Engineer advancing multimodal reinforcement learning, reward modeling, and scalable training infrastructure. Researching intelligent systems for Tether’s global blockchain-based digital finance ecosystem.
Enterprise Account Executive – AI Research SaaS
Enterprise Account Executive selling an AI-moderated research platform to major U.S. brands. Owning complex SaaS sales cycles from prospecting through enterprise close.
Principal AI Research Cybersecurity Scientist
Principal AI cybersecurity scientist researching frontier models, benchmarks, and security evaluations. Developing practical safeguards for Alice’s AI trust, safety, and security customers.
Senior Responsible AI Researcher
Senior Responsible AI Researcher advancing frontier-model safety, evaluation, and adversarial robustness research. Building benchmarks and safeguards for Alice’s AI trust and security solutions.
Master's Research Fellow, Conversational AI Research, Voice Analytics
Mestre pesquisando IA conversacional e análise de voz para o projeto de inovação do Itaú. Avaliação experimental, especificações técnicas e apoio à evolução de produtos de IA.
TA Manager – AI, Research and Innovation
GE Vernova Talent Acquisition Manager leading AI and innovation recruiting strategy across functions and geographies. Building talent pipelines, advising leaders, and developing high-performing recruiting teams.
Intern – Gen AI Research Engineer
Gen AI Research Engineer Intern researching Generative AI and AGI for healthcare treatment, payment, and operations. Developing LLMs, ML tools, databases, and governance solutions at Cotiviti.
Staff AI Scientist, Foundation Models
Staff AI Scientist developing multimodal foundation models for Tempus’s precision medicine platform. Advancing clinical and molecular AI from research through deployment across healthcare and life sciences.
Web Researcher – AI Benchmarking
Web Researcher designing difficult, evidence-traced benchmarks for frontier AI browsing agents. Building reproducible open-web investigations and validation documentation.
Web Researcher – AI Benchmarking
Web Researcher designing difficult open-web problems for frontier AI browsing agents. Building evidence trails and reproducible benchmarks from verifiable facts.
Applied AI Research Intern
Applied AI Research Intern evaluating AI tools across Vulcury’s consulting practice and ventures. Delivering evidence-based adoption, build, partnership, and competitive recommendations to the CEO.
Senior AI Research Scientist, Model-based RL
Senior AI Research Scientist developing model-based reinforcement learning for Phaidra’s AI-powered industrial automation systems. Designing safe agents, world models, and planning controllers for real-world deployment.
Staff Machine Learning Researcher
Staff Machine Learning Researcher advancing deep-learning advertising technology at RTB House. Designing production models, researching bidding solutions, analyzing A/B tests, and mentoring ML researchers.
Applied AI Scientist
Applied AI Scientist building production AI systems for Vantor’s spatial intelligence platform. Transforming geospatial data, imagery, and multimodal models into actionable Earth intelligence.
AI Research Engineer 6 – TL, Algo Core – AI for Member Systems
AI Research Engineer leading Netflix’s reward modeling and multi-objective optimization systems. Shaping promotional decisions across recommendation, personalization, and member experiences.
Senior Data Scientist – AI Research
Sr. Data Scientist advancing AI/ML initiatives for Lyra Health’s evidence-based mental healthcare platform. Building predictive and generative models, data pipelines, and evaluation frameworks for patients, providers, and employers.
US Lawyer – AI Legal Research, Evaluation
US-qualified Lawyer needed for legal research in tech project focusing on AI. Conducting analysis and drafting legal content for various US law topics in a remote capacity.
Senior Threat Researcher – AI, Autonomous Cybercrime
Senior Threat Researcher examining AI misuse in cybercrime for a cybersecurity SaaS firm. Engaging in proactive threat intelligence and producing original research.
Senior Speech AI Researcher
Senior Speech AI Researcher transforming unique data streams into the gold standard for AI training. Bridging mobile users and AI labs, defining research agenda and leading evaluation pipelines.
Head of Market Research – AI-Native Research
Head of Market Research at Maze, defining AI-native methods and building a research organization.