PhD Research Fellow – LLMs, Agents, Multi-Agent Systems, AI Reliability

Posted 6hrs ago

Employment Information

Education
Salary
Experience
Job Type

Report this job

Job expired or something wrong with this job?

Job Description

Pesquisador bolsista desenvolvendo pipelines, métricas e evals para agentes LLM e multiagentes. Projeto de inovação do Instituto Itaú de Ciência, Tecnologia e Inovação com Inova Talentos.

Responsibilities:

  • Map agent execution flows and identify key points for log collection and monitoring
  • Develop Python pipelines for collecting, storing, and processing execution logs
  • Define and implement metrics to evaluate agent performance, quality, and consistency
  • Build automated evaluation routines (evals) for different scenarios and use cases
  • Develop analyses and reports on agent behavior, identifying failures and opportunities for improvement
  • Support the implementation of dashboards or monitoring tools
  • Document the architecture, metrics, and results obtained throughout the project
  • Work on an innovation project at the Itaú Institute of Science, Technology and Innovation in partnership with Inova Talentos

Requirements:

  • Program: TECH 4
  • PhD
  • Degree in Data Science, Statistics, Computer Engineering, Computer Science, or a related field
  • Python programming
  • Ability to organize and reproduce a software project
  • Experience with LLMs beyond casual use
  • Knowledge of multi-agent systems and how to orchestrate them
  • Ability to extract, validate, and compare information in documents
  • Ability to define and interpret evaluation metrics
  • Experience instrumenting systems with logs, traces, and metrics
  • Ability to explain failures from a technical perspective
  • Ability to read articles and research papers, turn ideas into experiments, and clearly write and present results
  • Availability to work 40 hours per week
  • Previous applied research experience with AI, ML, or LLMs
  • Ability to build and evaluate end-to-end observable systems
  • Ability to transform an operational workflow into an agent architecture
  • Clear scientific writing and strong verbal communication skills
  • Autonomy and experimental maturity

Benefits:

  • Fellowship stipend: BRL 11,000.00
  • Work arrangement: Remote
  • Commitment: 40 hours per week
  • Duration: 12 months