Head of AI Engineering
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Head of AI Engineering building the Agent SDK and production AI systems for AIOS’s full-stack AI doctor. Leading agent architecture, evaluations, safety, reliability, model economics, and the Applied AI team.
Responsibilities:
- Build the AIOS Agent SDK as the foundation for world-class agents across the company
- Separate the reusable foundations of the existing customer-support harness from its customer-support logic and turn them into an opinionated internal platform
- Own agent architecture, model strategy, evaluations, AI reliability, technical safety, provider relationships, and the shared runtime
- Ensure the Agent SDK runs in production and provides context, tool, safety, evaluation, and observability infrastructure
- Turn Jesse’s harness into the platform powering Jesse, Aegis, and future AIOS agents
- Own the technical direction of Jesse and collaborate with engineers and the Clinical Product team building Aegis
- Build benchmarks using deterministic checks, simulations, model-based graders, human judgment, and production outcomes
- Convert production traces, incidents, tool failures, escalations, and outcomes into improved evaluations, architecture, models, and platform capabilities
- Implement authorization, validation, idempotency, auditability, recovery, human handoff, compliance, privacy, security, and regional requirements
- Own model selection, routing, fallbacks, caching, fine-tuning, deployment, and approximately $200k monthly model spend
- Own production runtime reliability, tracing, observability, testing, provider resilience, capacity, and incident response
- Set AI architecture and strategy, make major technical decisions, guide engineers, write production code, and build key foundations
- Inherit one engineer and grow the Applied AI team to approximately five people during the first year
- Own technical relationships with leading model providers and represent AIOS externally
Requirements:
- 8+ years of software engineering experience and active production contribution
- At least a bachelor’s degree in Computer Science, Machine Learning, or a closely related technical field
- Personally built and shipped an exceptional agentic system used by real customers
- Deep knowledge of harnesses, orchestration, context construction, retrieval, memory, state, tool design, structured workflows, and error recovery
- Experience building or meaningfully owning evaluation systems for probabilistic products
- Strong systems-engineering fundamentals, including APIs, distributed systems, concurrency, queues, databases, observability, failure modes, and production reliability
- Strong judgment around authorization, validation, idempotency, state transitions, auditability, recovery, and escalation for consequential agent actions
- Understanding of current frontier and open-weight model capabilities and limitations
- Technical depth to lead fine-tuning and AIOS-controlled deployment of open-weight models
- Successfully led and managed a small technical engineering team
- Ability to make difficult technical decisions, explain trade-offs, and challenge unsound approaches
- Ability to explain difficult technical ideas to engineers, product leaders, clinicians, and executives
- Independence in ambiguous environments
- Ability to write production code and lead from inside the work
- Ownership of outcomes when quality drops, costs spike, tools fail, or providers degrade
- Nice to have: experience with agent platforms, customer agents, high-stakes systems, model adaptation, long-term memory, real-time systems, provider relationships, talent development, research translation, and production debugging
Benefits:
- Early stage equity
- Comprehensive medical insurance (if appropriate)
- PTO with a yearly minimum (≥2wks/yr + local national holidays)
- Fully remote work
- Personal development budget for books, courses, coaching ($1200/yr)
- Personal wellness budget for gym, health apps ($1200/yr)
- Free biweekly health coaching
- Macbook & work-from-home equipment provided as needed
- Opportunity to shape the culture
Similar Jobs
Manager, Detection Engineering – Rapid Response Team
Detection engineering manager leading SentinelOne’s Rapid Response Team for AI-native cybersecurity. Developing threat detections, managing senior engineers, and improving rapid-response operations.
Head of Engineering Services
Head of Engineering Services scaling Virtuos’ client-facing game engineering services business. Defining offerings, growing global teams, and driving commercial performance and delivery governance.
Engineering Manager
Engineering Manager leading courier dispatch, matching, pricing, and ETA systems at Mrsool, an on-demand delivery platform. Guiding backend engineering, data science integration, and operational reliability at scale.
Engineering Manager
Engineering Manager leading engineers and hands-on development for Second Nature’s resident-experience platform. Shaping scalable, secure software for property managers and residents.
Head of Engineering
Head of Engineering leading technology and high-performing teams for a startup creating safer online experiences for millions of women in Brazil. Defining strategy, architecture, and scalable systems.
AI Engineering Manager
AI Engineering Manager at InPost driving AI engineering strategy and leading multidisciplinary teams. Shaping AI-native products with a focus on collaboration and innovation.
Senior Engineering Manager
Manage and lead multiple engineering teams to deliver high-quality software products. Drive engineering culture and technical vision at Civica, a GovTech champion supporting critical public services.
Engineering Manager
Manager, Engineering responsible for engineering management and hands-on coding for Beyond's products. Leading a team of engineers to enhance software quality and delivery.
Engineering Manager
Engineering Manager at Beyond overseeing engineering practices and team collaboration on revenue management solutions. Guiding architectural decisions with emphasis on AI and CI/CD practices.
Product Sales Manager, Software
Product Sales Manager responsible for growing digital manufacturing solutions portfolio. Collaborating with dealer network and regional sales managers to improve productivity and operational performance.
Senior Engineering Manager, Activation & Engagement
Senior Engineering Manager leading a team of engineers in shopper engagement at Instacart. Overseeing product lifecycle and technical direction while managing talent development.
Chemical Engineering Management Consultant – EU
Chemical Engineering Consultant working with refineries in the EU. Analyzing data and improving refinery operations through process engineering and consulting efforts.
Engineering Manager, Recording Platform
Engineering Manager overseeing the team responsible for Fellow's recording platform. Ensure quality and reliability while managing and developing engineers across multiple systems.
Engineering Manager – Catalog
Engineering Manager for Catalog team overseeing product data models and engineering leadership. Player-coach role requiring both technical involvement and team management.
Associate Product Manager – Software and Databases
Associate Product Manager at Wiley Science Solutions supporting the development of online products and scientific software. Collaborating cross-functionally to gather requirements and track product engagement metrics.
Staff Engineering Product Manager
Product Manager overseeing AI agent strategy and product development at Cisco. Focusing on improving operational capabilities and collaborating with cross-functional teams.
BAS Engineering Manager
BAS Engineering Manager leading projects on building automation systems at DLB Associates. Overseeing project deliverables, standards development, and client interaction with significant industry experience.
Senior Engineering Manager, Data Processing Platform
Engineering Manager leading Data Processing Platform team at Horizon3, transforming pentest data processing into real-time architecture for cybersecurity solutions.
Engineering Manager
Engineering Manager overseeing a team of engineers at NexusOne, a converged data platform for the AI era. Focused on team growth, performance, and product management collaboration.
Manager, Software Engineering
Manager of Software Engineering at Rush Street Interactive, leading software engineering teams for online casino and sports betting product delivery.