Technical Support Engineer – Slurm

Posted 13hrs ago

Employment Information

Education
Salary
Experience
Job Type

Report this job

Job expired or something wrong with this job?

Job Description

Technical Support Engineer supporting NVIDIA customers with production Slurm AI and HPC clusters. Diagnosing scheduler, Linux, networking, storage, authentication, database, and GPU infrastructure issues.

Responsibilities:

  • Own Slurm support cases from initial investigation through resolution for customers running production AI and HPC clusters
  • Diagnose complex problems involving slurmctld, slurmd, slurmdbd, job scheduling, node management, resource allocation, accounting, authentication, and high availability
  • Solve Slurm configuration and policy issues involving partitions, reservations, priorities, fair-share, quality of service, backfill, preemption, GRES/TRES, cgroups, and job constraints
  • Investigate performance, reliability, and scalability issues using logs, diagnostic data, configuration analysis, reproductions, and source-level debugging when required
  • Isolate problems across Slurm and dependencies including Linux, MUNGE, databases, networking, parallel storage, containers, GPUs, and cluster-management systems
  • Advise customers on Slurm configuration, upgrades, operational practices, resource management, and safe recovery from production incidents
  • Collaborate with engineering teams through clear technical descriptions, reproducible test cases, and well-supported defect reports
  • Develop guides, knowledge-base articles, diagnostic tools, and internal training to strengthen Slurm expertise across the support organization

Requirements:

  • BS degree in Computer Science, Engineering, or a related field, or equivalent practical experience
  • 5+ years of hands-on experience administering and supporting Slurm in production HPC or AI environments, including business-critical outage incidents
  • Expert-level understanding of Slurm architecture, daemons, configuration, scheduling behavior, accounting, resource management, and failure modes
  • Ability to independently identify sophisticated Slurm incidents and guide them to technically sound resolutions
  • In-depth Linux system-administration and troubleshooting experience, including systemd, cgroups, authentication, networking, and database-backed services
  • Experience operating Slurm across multi-user clusters with complex scheduling policies and heterogeneous compute resources
  • Strong analytical and research skills, including distinguishing Slurm defects from configuration, integration, infrastructure, and workload problems
  • Excellent written and verbal communication skills, including turning detailed technical findings into clear explanations and actionable recommendations
  • Experience supporting large-scale Slurm environments containing thousands of nodes or GPUs
  • Experience diagnosing scheduler performance, job-throughput, controller-load, and database-scaling issues
  • Familiarity with Slurm source code, plugins, SPANK, Lua job-submit plugins, or upstream issue investigation
  • Experience with containers and HPC integration technologies such as Pyxis, Enroot, Apptainer, or Singularity
  • Previous experience integrating Slurm with NVIDIA Base Command Manager, Bright Cluster Manager, or another cluster-management platform

Benefits:

  • Diverse, supportive work environment
  • Opportunities to make a lasting impact on the world
  • Full-time employment