Staff AI Builder
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Staff AI Builder designing and shipping production GenAI systems for Robots and Pencils’ enterprise clients. Leading agentic AI, RAG, AWS architecture, reliability, and technical mentoring.
Responsibilities:
- Lead the design and delivery of AI/ML systems
- Define AI architecture and lead model development and optimization
- Design and build agentic workflows, including reasoning loops, tool/function calling, and single- and multi-agent orchestration
- Build and maintain RAG pipelines with chunking, embeddings, OpenSearch vector search, re-ranking, and refresh handling
- Integrate AWS Bedrock and Agent Core, including MCP-based tool design
- Write and iterate on production system prompts
- Build evaluation and observability into agents using golden datasets, RAGAS-style metrics, LLM-as-judge, and tracing
- Design for LLM failures using retries, backoff, circuit breakers, fallback models, and user-facing degradation handling
- Build backend services in Python and Node.js, including AWS Lambda and API Gateway serverless architectures
- Design DynamoDB single-table schemas for conversation state, agent memory, and session history
- Support event-driven orchestration with Step Functions, SQS, and EventBridge
- Contribute to frontend integration points and write clean, tested code across the stack
- Support deployment, monitoring, and production troubleshooting in AWS and Docker environments
- Partner with product, design, and delivery leads across global teams to scope and ship features end-to-end
- Mentor engineers and improve agentic engineering practices, including use of Claude Code
- Own features and releases end-to-end, including debugging, hardening, and production reliability
Requirements:
- 6+ years of professional software engineering experience, including meaningful time shipping GenAI/LLM-powered systems in production
- Hands-on depth in agentic AI, including reasoning loops, tool/function calling, and multi-agent orchestration
- Practical RAG expertise, including chunking strategies, embeddings, vector databases such as OpenSearch or similar, cosine similarity search, and re-ranking
- Experience building evaluation and observability for LLM systems, including golden datasets, LLM-as-judge, RAGAS or comparable metrics, and LangFuse/LangSmith or similar tracing tools
- Strong prompt engineering skills
- Hands-on experience with the AWS GenAI stack: Bedrock, Agent Core, Lambda, DynamoDB single-table design, S3, SQS, EventBridge, and Step Functions
- Strong Python and Node.js skills
- Experience building full-stack applications and RESTful APIs
- Understanding of production reliability patterns for LLM-backed systems, including retries/backoff, circuit breakers, and fallback models
- Experience with Docker and cloud-native deployment
- Ability to own ambiguous, integration-heavy problems and provide technically deep answers
- Helpful extras: direct Bedrock Agent Core experience; regulated or high-stakes domain experience; workflow orchestration familiarity; production LLM observability experience
Similar Jobs
AI Safety Expert – English, Kannada
AI Safety Experts red-teaming conversational models for Mercor, which partners with AI labs to train frontier systems. Probing vulnerabilities and producing reproducible safety data.
AI Safety Expert – English, Kannada
AI safety red teamer probing conversational AI models for Mercor, which partners with leading AI labs to train frontier systems. Creating vulnerability data, attack cases, and reproducible reports.
AI Safety Expert – English, Marathi
AI safety red-teamer probing Mercor’s frontier AI models with adversarial English and Marathi inputs. Generating vulnerability annotations, attack cases, datasets, and reproducible reports.
AI Safety Red Teamer
AI Safety Red Teamer designing adversarial prompts and uncovering vulnerabilities in Mercor’s frontier AI projects. Evaluating model behavior across high-risk domains and supporting AI safety research.
AI Benchmark Engineer, Native Language Specialist – Chinese
AI benchmark engineer building Chinese-language Terminal-Bench tasks for Lilt’s multilingual AI platform. Validating coding agents, reference implementations, and deterministic verifiers.
AI Benchmark Engineer, Native Language Specialist – Chinese
AI benchmark engineer creating Chinese-language Terminal-Bench tasks for Lilt’s multilingual AI platform. Validating coding agents, verifier scripts, and multilingual software robustness.
AI Benchmark Engineer, Native Language Specialist – Arabic
Arabic AI benchmark engineer designing Terminal-Bench tasks for Lilt’s multilingual AI and human-verified language services. Building multilingual assets, verifiers, and calibration workflows for coding agents.
AI Benchmark Engineer, Native Language Specialist – Arabic
Arabic-speaking benchmark engineer designing Terminal-Bench tasks for LILT’s multilingual AI evaluation suite. Building datasets, verifiers, and calibrated tests for coding agents.
AI Safety Experts – English, Telugu
AI safety red-teamer probing conversational models for Mercor’s human-data AI training projects. Generating vulnerability datasets, attack cases, and reproducible reports to strengthen frontier AI systems.
AI Safety Expert – English, Tamil
AI red-team expert probing conversational models for Mercor, which partners with AI labs to train frontier systems. Generating adversarial data, vulnerability reports, and reproducible attack cases.
AI Safety Expert – English, Kannada
AI safety red teamer probing Mercor’s frontier AI models for jailbreaks, bias, and vulnerabilities. Generating reproducible human-data artifacts for AI labs and enterprise customers.
AI Safety Red Teamer
AI Safety Red Teamer identifying vulnerabilities in frontier AI systems for Mercor. Designing adversarial prompts and evaluating high-risk model behavior.
AI Safety Expert – English, Tamil
AI red team expert probing conversational models for Mercor, which partners with AI labs to train frontier systems. Testing vulnerabilities and producing actionable safety data.
Associate Director – Technology, AI, Digital Solutions – Healthcare
Associate Director leading healthcare technology, AI, and digital transformation engagements for BRG’s healthcare advisory practice. Translating client priorities into technology roadmaps and scalable solutions.
AI Strategist
AI Strategist helping retail and consumer brands identify operational opportunities and deploy applied AI systems. Owning client relationships, business cases, delivery direction, and measurable outcomes across Latin America.
AI Strategist
AI Strategist helping WON redesign retail and consumer business operations with applied AI. Owning client discovery, business cases, delivery direction, and measurable outcomes.
Senior AI Agent Engineer
Senior AI Agent Engineer building Rumor Avenue's new AI product from scratch. Developing Python/FastAPI backend, React interfaces, and production LLM features.
AI Safety Expert, English – Kannada
AI safety red-teaming specialist probing frontier models for Mercor. Generating adversarial test data, vulnerability analyses, and reproducible attack artifacts for AI labs and enterprises.
AI Safety Expert, English, Punjabi
AI safety red teamer probing Mercor-trained AI models through adversarial English and Punjabi testing. Generating vulnerability annotations, attack cases, datasets, and reproducible reports.
AI Safety Expert – English, Odia
AI safety expert red-teaming Mercor’s frontier AI models through adversarial testing and human-data generation. Evaluating vulnerabilities in English and Odia across flexible, fully remote contract projects.