Senior HPC & AMD GPU Infrastructure Engineer

Location
New York City or U.S

Employment Type
Full-time

Location Type
On-site, Hybrid, or Remote

Department
Infrastructure & Engineering

Compensation
• Competitive base salary
• Performance-based commission
• Equity

About Evergrid

Evergrid builds the infrastructure that powers advanced artificial intelligence at scale, with the Most Advanced Neocloud.

We design and operate critical GPU and power infrastructure for frontier AI workloads — environments where performance, reliability, and execution are non-negotiable. Our customers are building systems at the edge of what’s possible, and they depend on infrastructure that scales quickly and works under sustained pressure.

We care deeply about outcomes, ownership, and building durable systems and long-term partnerships.

The Role

As a Senior HPC & AMD GPU Infrastructure Engineer, you will take full ownership of the health, reliability, and performance of Evergrid’s AMD-only GPU compute clusters.

You’ll be the primary custodian of our high-density accelerator environments—working at the intersection of hardware operations, Linux systems engineering, distributed infrastructure, and machine learning workloads. This role spans everything from GPU bring-up and kernel-level debugging to maintaining and optimizing the AMD ROCm-based ML software stack that powers production-scale AI systems.

If you enjoy squeezing maximum performance out of hardware, debugging GPUs at scale, and building world-class AI infrastructure that actually ships, this role is for you.

What You’ll Do

System Health & Reliability (SRE)

• Serve as a primary on-call responder for system outages, GPU failures, node crashes, and cluster-wide incidents

• Work with POC and Active customers as the key point person for their GPU cluster.

• Diagnose and resolve issues quickly to minimize downtime and maintain SLA-level reliability

• Design and maintain monitoring for GPU health, thermals, PCIe topology, memory errors, and cluster load

• Coordinate with data center operators, hardware vendors, and on-site technicians for repairs, RMAs, and physical maintenance

Linux & Network Administration

• Install, patch, and maintain Linux systems (Ubuntu, CentOS, RHEL) across large GPU node fleets

• Own kernel tuning, OS configuration consistency, and fleet automation at scale

• Configure and maintain secure networking: VPNs, firewalls (iptables/firewalld), SSH hardening, and routing

• Manage identity and access systems (LDAP, FreeIPA, Active Directory)

• Administer and troubleshoot distributed storage systems (NFS, GPFS, Lustre)

AMD GPU & ML Stack Engineering (ROCm-first)

• Lead deployment and bring-up of new GPU nodes, including BIOS configuration, NUMA tuning, and topology validation

• Maintain and debug AMD GPU drivers, kernel modules, and the ROCm runtime stack across production fleets

• Own the AMD ML software stack: ROCm, PyTorch (ROCm builds), JAX (ROCm/XLA), RCCL, hipBLAS/hipDNN, MIOpen, and supporting runtimes

• Debug complex failures across GPUs, compilers, ML frameworks, and distributed training/inference systems (e.g., RCCL hangs, HIP memory faults, ROCm kernel crashes, framework build/link issues, vLLM build failures on ROCm)

• Partner closely with ML and platform teams to ensure infrastructure supports both research iteration and production reliability

What We’re Looking For

• 5+ years of experience in HPC, GPU cluster operations, Linux systems engineering, or similar roles

• Bachelor’s or Master’s degree in Computer Science, Computer Engineering, Electrical Engineering, or a related field

• Deep hands-on experience with AMD MI-series GPUs, including driver and kernel-level debugging

• Strong understanding of Linux internals, kernel modules, hardware bring-up, and performance tuning

• Experience securing and operating production infrastructure (VPNs, firewalls, SSH, identity systems)

• Proficiency in Bash and Python for automation, tooling, and operational workflows

• Strong familiarity with ML software stacks and runtime behavior in ROCm environments (ROCm/HIP, MIOpen, RCCL, PyTorch, JAX)

• Experience debugging high-performance networking and RDMA (e.g., InfiniBand or RoCE), plus cluster-level communication failures that impact distributed training

Nice to Have

• Experience with schedulers and orchestration systems (Slurm, Kubernetes)

• Exposure to model serving and inference optimization (e.g., vLLM, SGLang) on ROCm

• Hands-on experience with configuration management and IaC tools (Ansible, SaltStack, Terraform)

• Prior experience supporting ML research or production AI teams in startup or high-growth environments