Senior Software Engineer – AI Inference & Runtime Platform
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Senior Software Engineer building AI inference infrastructure and secure agent sandboxes at AZX. Operating Kubernetes, GPU serving, microVM isolation, and stateful data platforms.
Responsibilities:
- Own the inference control plane for open-weight models and custom task-model zoos across managed GPU clouds and customer-managed Kubernetes clusters
- Manage serving-tier engine deployment and configuration, cold-start strategy, per-model SLOs, and upgrade/canary discipline
- Administer the Kubernetes layer for inference and sandbox workloads, including operators, CRDs, autoscaling, GPU scheduling and sharing, and node lifecycle
- Own the stateful data plane for hosted vector stores and graph stores, including deployment, backups, scaling, recovery, and tested restores
- Oversee the sandbox runtime and host-side control plane, including lifecycle, exec, snapshot/fork, teardown, metering, and isolation-boundary threat modeling
- Direct FastAPI control-plane services, Terraform/OpenTofu, Bicep, and dashboards
- Run the platform layer supporting backend services and debug into backend services as needed
- Manage the open-source posture through reviewed PRs, documentation, and reproducible builds
- Build the fork engine, guest agent, and multi-substrate model lifecycle using Rust, Kubernetes controllers, and FastAPI endpoints
- Account for per-token costs accurately, including client mid-stream disconnections
- Operate hardware-isolated microVMs for secure, compliant execution of untrusted agent-generated code
Requirements:
- 5+ years of shipping production systems in a systems language
- Rust is the house language; deep Go, C/C++, or Zig with genuine appetite for Rust counts
- Familiarity with async runtimes, memory-safety discipline, and debugging at the syscall boundary
- Experience operating Kubernetes workloads, including controllers or operators, scheduling, autoscaling, and node lifecycle
- Ability to threat-model isolation boundaries, including namespaces, cgroups, seccomp, and hypervisors
- Experience applying security best practices for agentic execution: least privilege, no credentials in the sandbox, audit trails, and human approval on write actions
- Hands-on experience deploying or operating open-weight LLM serving infrastructure such as vLLM/SGLang or similar
- Performance discipline in distributed systems
- Practical depth in Rust (tokio), Python/FastAPI, Kubernetes operators, KEDA, Karpenter, GPU device plugins/DRA
- Familiarity with Firecracker, Kata, gVisor, or comparable isolation technology
- Familiarity with secrets management and egress control, such as Vault/KMS-class tools
- Experience hosting stateful systems including vector stores and graph stores, with backup and failover discipline
- Comfort operating across AWS/Azure/GCP and managed GPU clouds
- Experience with Terraform/OpenTofu, Bicep, and OpenTelemetry
- Bachelor's Degree; Master's is a plus
- Must be currently authorized to work in the United States on a full-time basis
- Must be able to travel 2x/year for company summits
Benefits:
- Competitive early-stage startup compensation (based on capabilities, experience, and location)
- Bonus eligibility
- Health insurance with meaningful coverage for dependents
- Flexible paid time off
- Equity
- Fully remote culture with a cluster of teammates in Seattle
- Company summits 2x/year
Similar Jobs
AI Engineer – AI Model Integration Build
AI Engineer building a serverless AWS model-integration layer for Cloudary’s multi-cloud AI filmmaking platform. Connecting Bedrock, ModelArk, video, and image providers without disrupting the existing Firebase/GCP frontend.
AI Engineer
AI Engineer building Azure, Generative AI, and RAG solutions for Expleo’s global engineering and technology services. Integrating intelligent automation into business applications.
Backend AI Engineer
Backend AI Engineer building scalable model-serving, RAG, and microservice infrastructure. Powering Nexxa.AI’s autonomous AI systems for manufacturing, infrastructure, logistics, and legacy environments.
AI Engineer – Anthropic Campus Hire
Early-career AI Engineer building Claude-powered applications, agentic workflows, and RAG solutions. Supporting NewRocket’s enterprise ServiceNow consulting and AI implementations.
AI Engineer
AI Engineer building reliable GenAI, RAG, and agentic systems for Red Nucleus’s life sciences solutions. Delivering AI-generated insights for scientific and medical writing teams.
Associate Product Manager – Platform AI
Associate Product Manager advancing applied AI features across Acumatica’s cloud ERP platform. Supporting product planning, market analysis, launches, customer discovery, and AI enablement.
Process Engineer, AI in Practice – Early Career
Engenharia de processos e IA generativa para automações na agência F2F. Mapeamento, prototipagem e validação de soluções internas com os sócios fundadores.
AI Engineer
Senior AI Engineer building Azure-powered enterprise AI solutions for a global IT services provider. Developing LLM, RAG, and agentic AI applications with .NET and Python.
Artificial Intelligence Engineer
Artificial Intelligence Engineer building Copilot-powered automation, RAG, and embedded AI for Emergifi’s credit-union platform. Delivering secure, compliant solutions that improve member experiences and operations.
AI Engineer III
AI Engineer building speech, LLM, and multilingual retrieval tools for Propio’s real-time interpretation services. Deploying practical AI systems, summaries, and interpreter QA workflows.
Senior Agentic, AI Engineer
Senior Agentic AI Engineer building production agents for Worth AI’s regulated-finance automation. Owning retrieval, evaluations, tools, MLOps, and compliant deployment.
AI Engineer – .NET, Azure AI
AI Engineer building .NET/C# backend APIs and Azure AI solutions at Imaginary Cloud, a Portugal-based software development and UX/UI design company. Deploying generative AI models into production.
AI Creative Engineer – AI Video Ads
AI Creative Engineer creating performance video ads with generative models for Created AI. Refining prompts, workflows, and model selection for real-brand campaigns.
Associate Product Manager – Platform AI
Associate Product Manager advancing applied AI capabilities for Acumatica’s cloud ERP platform. Supporting product planning, market analysis, releases, customer discovery, and AI feature enablement.
Software Engineer – AI Systems, Go
Software Engineer building reliable production AI agents and distributed workflows in Go. Shaping Stanbridge University’s architecture for scalable, measurable AI systems.
Staff Software Engineer, AI Platform
Staff Software Engineer building JLL’s internal AI platform and model gateway. Enabling AI agents and products across a global real estate and investment management company.
Senior Product Manager – SaaS Platform, AI Focus
Senior Product Manager owning AI-focused SaaS product strategy for Magicline’s gym-management platform. Leading discovery, roadmaps, and cross-functional delivery for fitness businesses.
Senior Product Manager – SaaS Platform, AI Focus
Senior Product Manager owning AI-focused SaaS product strategy and delivery. Shaping tools for Magicline’s cloud-based gym-management platform serving fitness businesses across Europe.
Mid-level AI Engineer, MLOps, AWS
Engenheiro de IA levando Machine Learning e IA Generativa à produção na Leega. Construindo pipelines MLOps escaláveis, seguros e monitorados no ecossistema AWS.
Senior Data & AI Engineer – AI Agent
Engenheiro de Dados e IA sênior desenvolvendo agentes multiagentes para consultoria brasileira de dados. Integrando LLMs, MCP, APIs e plataformas corporativas para o setor de pagamentos.