Applied AI Engineer
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Senior Applied AI Engineer developing resilient AI systems. Focusing on production-grade stability and optimization of LLM-powered workflows for LATAM.
Responsibilities:
- Design, build, and maintain evaluation pipelines for production AI agent systems
- Instrument multi-agent workflows with tracing and observability tooling
- Build evaluation datasets using real production traffic and interaction logs
- Develop quality scoring and robustness scoring systems for LLM outputs
- Improve reliability of AI systems handling non-deterministic model behavior
- Implement and optimize HITL (Human-in-the-Loop) escalation workflows
- Analyze production failures and drive architectural improvements
- Own the full feedback loop between evaluations, prompt optimization, architecture updates, and re-testing
- Contribute to prompt engineering and model optimization strategies
- Collaborate on multi-agent orchestration and workflow reliability decisions
- Work across backend systems, deployment pipelines, monitoring, and operational sustainment
- Participate in production support and on-call responsibilities
- Maintain high engineering standards around scalability, observability, and maintainability
- Operate independently across development, testing, deployment, and production ownership
Requirements:
- 5+ years of backend or AI engineering experience in production environments
- Strong hands-on experience with production LLM or agentic AI systems
- Proven experience debugging and maintaining non-deterministic AI workflows under live traffic
- Experience building or operating evaluation/evals pipelines for AI systems
- Strong understanding of scorer design, feedback loops, and AI system evaluation methodologies
- Excellent Python backend engineering skills
- Production experience with: FastAPI Django Celery LangGraph or similar orchestration frameworks
- Experience with observability and tracing tools such as: Langfuse Grafana Loki OpenTelemetry or equivalent
- Experience deploying and operating distributed backend systems
- Strong understanding of AI reliability, prompt behavior, and model failure handling
- Ability to independently own projects end-to-end
- Experience working in asynchronous remote teams
- Strong written communication skills in English.
Benefits:
- Fully Remote
- LATAM-friendly collaboration preferred
Similar Jobs
Senior Applied AI Engineer
Engineering role enhancing AI workflows and systems for Triquetra Health's innovative AI applications. Collaborating with marketing, R&D, and operations to deliver reliable solutions.
Principal Applied AI Engineer, Finance
Principal Applied AI Engineer leading design and delivery of AI systems transforming financial decision-making. Overseeing advanced predictive models and software engineering practices in finance context.
Technical Product Manager, AI Engineering – Systems
Technical Product Manager leading end-to-end development of AI platform and customer experiences at Jerry.ai, an innovative super app simplifying car ownership.
Senior Technical Product Manager, AI Engineering – Systems
Technical Product Manager driving AI platform and customer experiences at Jerry.ai. Leading development from the initial roadmap to global rollout in car ownership solutions.
Lead Gen AI Engineer
Lead Generative AI initiatives at Orion Innovation, delivering AI-driven solutions across various industries and mentoring teams. Collaborate with stakeholders while ensuring best practices in AI development.
Senior Director – AI Engineering
Director of AI Engineering leading innovation and strategic growth across FICO's platform. Working at the forefront of Generative AI efforts enhancing fraud detection and decision automation.
AI Engineer, Agentes de Software
AI Engineer designing and implementing multi-agent architectures for a financial technology company. Collaborating on specialized agent orchestration and contextual management mechanisms.
Senior Manager – AI Engineer
Senior Manager in AI engineering at PwC leading design and development of AI-driven applications and microservices. Collaborating with cross-functional teams to innovate processes and drive operational excellence.
Product Owner – Agentic AI Platform
Product Owner leading AI-powered features for digital advertising platforms. Collaborating with cross-functional teams and leveraging LLMs for innovative user experiences.
AI Engineer
AI Engineer transforming business context into real deliveries using AI agents at Gauge, a digital marketing company.
AI Architect
AI Architect at Gauge, developing Agent Skills for AI tasks and collaborating across projects. Focus on supporting multiple teams with efficient AI infrastructure and documentation.
Senior Manager – AI Architect, Cyber Architecture, OT and Engineering
Senior Manager – AI Security / Cyber AI Architect leading AI Security practice at EY. Driving enterprise adoption of secure AI architectures and mentoring teams in a global environment.
Staff AI Engineer
Staff AI Engineer creating autonomous systems to streamline insurance workflows for brokers. Focus on building advanced AI solutions and mentoring talent in a flexible remote environment.
Director of Data, AI Engineering
Director of Data & AI Engineering leading a team for data initiatives at Interplay Learning, focusing on product innovation and organizational efficiency in EdTech.
Principal AI Engineer
Principal AI Engineer designing generative AI solutions for mental healthcare at Rula. Hands-on role focusing on architecture, applied research, and system design.
AI Engineering Intern
AI Developer Summer Intern at PHIL, a health-tech startup improving prescription management with AI solutions. Collaborate with cross-functional teams on impactful AI initiatives.
AI Architect
AI Architect role focused on designing and delivering production AI systems for enterprise clients with Cyclotron. Collaborating with senior architects and contributing to AI solutions across platforms.
Senior AI Architect, Consultant
Senior AI Architect at MAI Group designing impactful AI architectures for regulated industries. Engage in hands-on development of AI systems with direct client interactions.
Forward Deployed AI Engineer
AI Engineer developing AI solutions for operational processes at Accelerant. Collaborating with stakeholders to automate workflows and improve efficiency in financial services.
AI Engineer
AI Engineer at Cyclotron assisting in building and deploying AI-driven applications using Microsoft technologies. Collaborating with teams to deliver intelligent solutions and optimize data workflows.