Staff Software Engineer, GPU Infrastructure – HPC
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Staff Software Engineer specializing in GPU Infrastructure, collaborating on cutting-edge AI workloads to elevate model performance and efficiency at Cohere.
Responsibilities:
- Build and scale ML-optimized HPC infrastructure: Deploy and manage Kubernetes-based GPU/TPU superclusters across multiple clouds, ensuring high throughput and low-latency performance for AI workloads.
- Optimize for AI/ML training: Collaborate with cloud providers to fine-tune infrastructure for cost efficiency, reliability, and performance, leveraging technologies like RDMA, NCCL, and high-speed interconnects.
- Troubleshoot and resolve complex issues: Proactively identify and resolve infrastructure bottlenecks, performance degradation, and system failures to ensure minimal disruption to AI/ML workflows.
- Enable researchers with self-service tools: Design intuitive interfaces and workflows that allow researchers to monitor, debug, and optimize their training jobs independently.
- Drive innovation in ML infrastructure: Work closely with AI researchers to understand emerging needs (e.g., JAX, PyTorch, distributed training) and translate them into robust, scalable infrastructure solutions.
- Champion best practices: Advocate for observability, automation, and infrastructure-as-code (IaC) across the organization, ensuring systems are maintainable and resilient.
- Mentorship and collaboration: Share expertise through code reviews, documentation, and cross-team collaboration, fostering a culture of knowledge transfer and engineering excellence.
Requirements:
- Deep expertise in ML/HPC infrastructure: Experience with GPU/TPU clusters, distributed training frameworks (JAX, PyTorch, TensorFlow), and high-performance computing (HPC) environments.
- Kubernetes at scale: Proven ability to deploy, manage, and troubleshoot cloud-native Kubernetes clusters for AI workloads.
- Strong programming skills: Proficiency in Python (for ML tooling) and Go (for systems engineering), with a preference for open-source contributions over reinventing solutions.
- Low-level systems knowledge: Familiarity with Linux internals, RDMA networking, and performance optimization for ML workloads.
- Research collaboration experience: A track record of working closely with AI researchers or ML engineers to solve infrastructure challenges.
- Self-directed problem-solving: The ability to identify bottlenecks, propose solutions, and drive impact in a fast-paced environment.
Benefits:
- An open and inclusive culture and work environment
- Work closely with a team on the cutting edge of AI research
- Weekly lunch stipend, in-office lunches & snacks
- Full health and dental benefits, including a separate budget to take care of your mental health
- 100% Parental Leave top-up for up to 6 months
- Personal enrichment benefits towards arts and culture, fitness and well-being, quality time, and workspace improvement
- Remote-flexible, offices in Toronto, New York, San Francisco, London and Paris, as well as a co-working stipend
- 6 weeks of vacation (30 working days!)
Similar Jobs
Staff Product Engineer
Product Engineer at Ashby focusing on building software solutions that enhance Talent Acquisition processes for various clients. Engage in end-to-end project ownership while innovating in the product space.
Staff Product Engineer
Product Engineer at Ashby developing innovative technology solutions for talent acquisition. Collaborating with teams to implement features from research to deployment, improving user experience and functionality.
Senior Product Engineer
Senior Product Engineer responsible for end-to-end feature development at a talent acquisition software startup. Collaborating on product specs, wireframes, and project ownership while managing a versatile tech stack.
Senior Product Engineer
Senior Product Engineer building innovative tools for Talent Acquisition teams in a versatile and supportive work culture. Lead projects end-to-end in a remote-first environment with high ownership.
Full Stack Engineer
Senior Full Stack Engineer responsible for core platform development and integrations at Panoptyc. Involves building high-performance systems for AI-driven retail loss prevention solutions.
Enterprise GTM Engineer
Enterprise GTM Engineer driving pipeline growth for MedTech sales teams with complex accounts. Combining org-mapping, consultative engagement, and AI-powered workflows for effective prospecting.
Senior Full-Stack Software Developer – Consumer
Senior Full-Stack Developer at gaiia, building platforms for Communications Service Providers. Design and develop APIs, collaborate with cross-functional teams, and solve complex UX challenges.
Senior Full-Stack Software Developer – CRM
Full-Stack Software Developer role focused on CRM at gaiia. Build and own critical systems in a remote environment with a team of developers.
Software Engineer
Software Engineer enhancing Ursa Health's flagship software and developing new applications. Working within AWS and Azure to improve data management and analytics solutions for healthcare.
Senior Full Stack Software Engineer
Senior Full Stack Engineer designing and delivering software solutions for government services. Collaborating in Agile teams to modernize legacy systems.
AI-Native Full Stack Engineer
Join Yia as an AI-native full stack software engineer helping small businesses leverage AI technology. Shape product development in a flexible part-time role in a startup environment.
Software Engineer, Applications
Software Engineer, Applications developing AI capabilities within CentralReach's healthcare software. Leading application development and collaborating with product and engineering teams.
Senior Software Engineer
Senior Software Engineer leading design and development of complex applications at CrossCountry Mortgage. Mentoring engineers and driving best practices in a collaborative environment.
AWS Tech Lead – Platform Operations
AWS Tech Lead managing platform operations and architecture at Supabase, ensuring secure and efficient AWS environments for global development teams.
Full Stack Developer
Full Stack Developer designing and deploying enterprise-grade applications integrating AI and data services at GDIT. Collaborate with cross-functional teams and mentor other developers in a dynamic environment.
Senior Embedded Software Developer
Sr. Embedded Software Developer driving technical direction and implementation for LynxOS-178 RTOS. Collaborating across teams to tackle complex technical challenges in embedded systems.
Senior Software Engineer
Develop backend integrations focusing on API contracts and automation for SysMap's diverse business segments. Collaborate on system integrations and ensure SRE reliability practices are implemented.
Senior Fullstack Developer
Senior Fullstack Developer designing and developing RESTful APIs utilizing .NET and ASP.NET Core at Brasil-Frenet. Contributing to frontend component development and end-to-end project delivery.
Senior Software Engineer
Senior Software Engineer helping to modernize a healthcare payment integrity platform using AI tools and .NET technologies. Collaborating with teams to deliver secure and efficient software solutions while enhancing system architecture and performance.
Staff Engineer
Staff Software Engineer responsible for developing fraud detection systems for Givebutter, a nonprofit fundraising platform. Collaborate with cross-functional teams to enhance security and operational integrity.