AI Engineer, Agent/Platform Tracks
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
AI Engineer developing specialized pharmacovigilance agents to enhance clinical research safety at Parexel. Collaborating with cross-functional teams to implement AI solutions using large language models.
Responsibilities:
- Implement the specialized pharmacovigilance agents: write system prompts, configure model parameters, build tool-use definitions, and define agent boundaries to ensure precise adverse event processing
- Build and iterate prompt chains for each processing step: source document parsing, field extraction, MedDRA coding suggestions, causality assessment logic, narrative drafting, and E2B(R3) output generation
- Develop the deterministic rule engine layer: implement ICH E2B field validation checks, MedDRA hierarchy verification, and regulatory logic constraints that operate alongside LLM outputs
- Create and maintain evaluation datasets in collaboration with the pharmacovigilance domain team: annotated ground-truth cases, edge case libraries, and regression test suites
- Develop and maintain Model Context Protocol (MCP) servers to expose enterprise applications, APIs, databases, and services as standardized tools for AI agents
- Implement secure MCP integrations, tool definitions, authentication, and testing to enable reliable agent interaction with internal and external systems
- Run accuracy benchmarks, analyze failure modes, and iterate on prompts and agent configurations to improve performance against defined thresholds
- Implement the quality control agent's cross-verification logic: configure separate Claude instances, build comparison algorithms, and calibrate confidence scoring
- Build human-in-the-loop feedback mechanisms: reviewer interfaces for accept/modify/reject decisions, structured feedback capture, and feedback-to-prompt-improvement pipelines
Requirements:
- 3+ years of software engineering experience, with at least 1 year building applications that use LLM APIs (Anthropic, OpenAI, or equivalent)
- Proficiency in Python with demonstrated experience in production environments
- Experience building and evaluating NLP or LLM-based systems with measurable quality metrics
- Strong problem-solving skills and ability to work independently while collaborating with team members
- Bachelor's degree in computer science, or a related field, or equivalent professional experience
- Strong prompt engineering skills and experience writing and iterating system prompts, few-shot examples, chain-of-thought patterns, and structured output formats
- Solid foundation in Python with hands-on experience in LLM orchestration frameworks such as LangChain, LangGraph, or similar tools
- Experience building evaluation pipelines for NLP or LLM outputs: precision/recall measurement, confusion matrices, and threshold tuning
- Comfort with AWS services including S3, Lambda, and IAM basics, with AWS Bedrock experience being a plus
- Excellent written communication skills: ability to document prompt design decisions, evaluation results, and agent behavior specifications for validation purposes
- A collaborative mindset and ability to work effectively with cross-functional teams including domain experts and platform engineers
Benefits:
- Flexibility, growth, and creating space for people to do their best work
Similar Jobs
Staff Engineer, AI Platform – Architecture
Staff Engineer responsible for guiding AI platform architecture and reusable component strategy at Shield AI. Leading engineering efforts for AI enablement and governance.
Senior Platform Engineer
Sr. Staff Platform Engineer developing internal developer platforms at eHealth. Leading AWS Cloud infrastructure optimization and AI/ML integration efforts to empower engineering teams.
Engineer II – Platform
Platform Engineer designing and building solutions for technology integration at OSC. Responsible for multi-cloud infrastructure, AI tooling, system architecture, and collaboration across teams.
Director – Platform & Infrastructure
Director of Platform & Infrastructure leading modern infrastructure initiatives for ShiftKey's SaaS and Marketplace engineering teams. Collaborating with distributed teams across various regions and time zones.
Senior Platform Engineer, Data Platform
Senior Platform Engineer joining the Data Platform team to manage identity and security services. Role involves backend work, Dgraph internals, and cross-team collaboration.
Principal Engineer I – Prepurchase Platform
Principal Engineer I specializing in building backend services for Ticketmaster's Prepurchase Platform. Writing code, solving problems, and modernizing systems for high-demand ticket sales.
Staff Platform Engineer
Staff Platform Engineer responsible for Moxie's infrastructure and incident response, utilizing DevOps and SRE practices to ensure platform reliability. Focus on cloud systems and CI/CD pipelines.
Senior Software Engineer, Identity Platform
Senior Software Engineer at Prove developing identity verification solutions. Collaborating with teams to build robust and high-performance applications.
Senior AIOps Platform Engineer
Senior AIOps Strategy & Implementation Engineer at EY optimizing observability platforms and building automation workflows in cloud and on-prem environments.
Principal Software Engineer, MDR Platform
Senior Software Engineer developing and maintaining core Golang applications for cybersecurity operations. Collaborating across teams and shaping architectural direction in a growing MDR platform.
Data Engineer – Platform
Data Engineer, Platform at DraftKings building and scaling data ingestion pipelines. Enhance customer experiences and streamline operations through innovative data solutions.
Platform Engineer
Platform Engineer supporting design and operation of Marathon’s cloud infrastructure. Collaborating with teams to implement automation, IaC, and maintain operational standards.
Senior Data Engineer – Data Platform
Senior Data Engineer building and operating foundational data infrastructure for analytics at SunStrong Management. Collaborates with teams to deliver production-grade solutions with quality controls and security.
Platform Engineer
Platform Engineer developing backend systems and infrastructure for financial platforms at Shift Markets. Building scalable systems with TypeScript and Python in a flexible remote environment.
Senior Mobile Engineer – Cross-Platform
Senior Mobile Engineer responsible for developing the Grove patient-facing mobile application. Collaborating with offshore teams and ensuring high-quality mobile experiences across iOS and Android.
Senior Software Engineer, Platform
Senior Software Engineer responsible for developing data-intensive APIs and maintaining infrastructure systems. Collaborate within a cross-functional team in a remote environment across Canada.
Platform Engineer – Karriereambitionen
Platform Engineer developing cloud projects at Auralis Group. Leading teams and managing cloud infrastructures with flexible remote working conditions.
Principal Engineer – Platform Engineering
Principal Engineer focusing on Kubernetes platform strategy and implementation at Cisco. Leading cloud-native infrastructure projects and mentoring senior engineering teams.
Staff Software Engineer, Platform
Staff Software Engineer leading infrastructure to enhance the AI platform at Cake. Responsible for designing, operating, and owning the ML platform systems end-to-end.
Senior Microsoft 365 Platform Engineer
Senior Microsoft 365 Platform Engineer managing Microsoft 365 collaboration, identity, and endpoint management platforms at Repligen. Responsible for engineering, administration, governance, and continuous improvement.