Member of Technical Staff, Inference
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Inference runtime engineer optimizing vLLM, the open-source AI inference engine. Improving LLM and diffusion-model serving across hardware and architectures.
Responsibilities:
- Push the boundaries of LLM and diffusion model serving
- Work at the core of vLLM to optimize model execution across diverse hardware and architectures
- Develop inference runtime innovations for mixture-of-experts, multimodal, and agentic architectures
- Implement inference techniques and model architectures from research papers
- Contribute performant and maintainable code
- Debug complex machine-learning codebases
- Directly improve how AI inference is run by making inference cheaper and faster
Requirements:
- Bachelor's degree or equivalent experience in computer science, engineering, or similar
- Deep understanding of transformer architectures and their variants
- Strong programming skills in Python with experience in PyTorch internals
- Experience with LLM inference systems such as vLLM, TensorRT-LLM, SGLang, or TGI
- Ability to read and implement model architectures and inference techniques from research papers
- Ability to contribute performant and maintainable code and debug in complex ML codebases
- Preferred: deep understanding of KV-cache memory management, prefix caching, and hybrid model serving
- Preferred: familiarity with RL frameworks and algorithms for LLMs
- Preferred: experience with multimodal inference across audio, image, video, and text
- Contributions to open-source ML or system infrastructure projects are preferred
- Bonus: core feature implementation in vLLM or other inference engine projects
- Bonus: contributions to vLLM integrations such as verl, OpenRLHF, Unsloth, or LlamaFactory
- Bonus: widely-shared technical blogs or side projects on vLLM or LLM inference
- Required application materials: resume, GitHub handle, and a link to a relevant personal project, open-source contribution, or technical blog post
Benefits:
- Competitive benefits appropriate to your location, including health coverage where applicable
- Equity
- Visa sponsorship on a case-by-case basis
- Fully remote work
- Timezone-flexible schedule with regular overlap with Pacific Time for critical syncs
Similar Jobs
Senior App Developer
Senior Mobile Developer leading Flutter and Dart app development for Zenzero’s managed IT, cyber security, cloud, and AI solutions. Modernising legacy codebases and mentoring developers.
Senior C# Developer – Explorers, Identity Resilience
Senior C# Developer building Veeam’s SQL, database, and Active Directory recovery products. Designing resilient REST/gRPC services, agents, PowerShell interfaces, and integration tests.
Senior C# Developer – Explorers, Identity Resilience
Senior C# Developer building Veeam’s recovery software for databases and Active Directory. Designing distributed REST/gRPC orchestration and long-running agent operations.
Quantitative Developer, Internship
Quantitative Developer intern building high-frequency trading systems for InfiniteQuant, a global quantitative trading and technology company. Developing optimized code, data pipelines, simulators, and monitoring software.
Sitecore Developer
Sitecore .NET Developer building and enhancing Sitecore applications for IT services clients. Configuring personalization and analytics to deliver scalable digital experiences.
Zuora Billing Developer
Billing Platform consultant implementing and customizing enterprise Quote-to-Cash solutions for IT services clients. Designing integrations, billing workflows, and technical delivery across business and technical teams.
Industrial Engineering Co-op
Industrial Engineering co-op improving healthcare processes and operations at Wellstar, Georgia’s integrated health system. Analyzing data and facilitating process improvement projects.
Software Engineer
Software Engineer developing, testing, deploying, and supporting Home Depot applications. Collaborating in agile teams while building technical skills across cloud, CI/CD, and microservices.
Senior Mobile SDK Engineer
Senior Mobile SDK Engineer building Unity SDK architecture, native bridges, and release tooling. Powering a mobile-gaming loyalty platform connecting players with games, friends, and rewards.
Senior Mobile SDK Engineer
Senior Mobile SDK Engineer building Unity gaming SDKs for MWDN’s gaming loyalty platform client. Owning SDK architecture, native mobile bridges, releases, testing, and partner experience.
Senior Mobile SDK Engineer
Senior Mobile SDK Engineer building Unity gaming SDKs for MWDN, a global IT outstaffing company. Owning SDK architecture, native bridges, releases, testing, and partner experience.
Senior Developer – Telecom Mobile Test Probes
Senior Developer building C++/Linux test probes for CSG’s telecom testing platform. Simulating 4G, 5G, and IoT mobile services for global operators.
Senior Tableau Developer
Senior Tableau Developer building governed Tableau analytics for Koniag’s federal government customers. Delivering secure dashboards, integrations, performance tuning, and platform sustainment.
DocuSign CLM Developer
DocuSign CLM Developer building document workflows and Salesforce integrations. Supporting M9 Solutions’ federal government IT services and contract management systems.
Senior Appian Developer
Senior Appian Developer modernizing complex enterprise applications and plugins for ICF, a global advisory and technology services provider. Leading Appian upgrades, integrations, Agile delivery, and emerging AI initiatives.
Senior Flutter Developer
Senior Flutter Developer architecting and maintaining high-performance Flutter applications for Somnio Software, Latin America's Flutter-focused development company. Translating UI/UX designs, integrating APIs, and optimizing mobile experiences.
SAP Data Services Developer
SAP Data Services Developer building BODS integrations for CACI’s DoD SAP S/4HANA transportation platform. Improving secure data quality and logistics operations for USTRANSCOM.
Desarrollador Power Platform Senior, Copilot
Desarrollador senior de Power Platform y Copilot Studio en Devoteam, partner estratégico de Microsoft. Creación de aplicaciones, automatizaciones, analítica e integraciones empresariales para clientes.
Mid-Level WordPress VIP Developer
WordPress VIP developer building secure, accessible, high-performing digital services. Supporting federal government agencies through platform development, integrations, testing, and operational documentation.
COBOL Developer
COBOL Developer maintaining and modernizing legacy mainframe applications for federal government agencies. Supporting COBOL, JCL, testing, production operations, and STIG compliance.