Helix AI Engineer, Training Infrastructure
@ FigureFigurefigure.com
Helix AI Engineer, Training Infrastructure
Posted 18 months ago
This job is still taking applications, but it's been up a while.
About the job
Figure develops autonomous humanoid robots for home and commercial use, aiming to deploy intelligent robots globally. The Helix team focuses on infrastructure, training systems, and scalable AI research tools to advance robot capabilities and deployment.
Requirements
- Strong software engineering fundamentals
- Bachelor's or Master's in CS or related
- Experience with Python and PyTorch
- Managing HPC clusters for deep learning
- 4+ years building backend systems
Qualifications
- Experience managing cloud infrastructure
- Knowledge of job scheduling and orchestration tools
- Familiarity with configuration management tools
Full job description
Figure is an AI robotics company developing autonomous general-purpose humanoid robots. The goal of the company is to ship humanoid robots with human level intelligence. Its robots are engineered to perform a variety of tasks in the home and commercial markets. Figure is headquartered in San Jose, CA.
Figure's vision is to deploy autonomous humanoids at a global scale. Our Helix team is looking for an experienced Training Infrastructure Engineer to take our infrastructure to the next level. This role is focused on managing the training cluster, implementing distributed training algorithms, data loaders, and developer tools for AI researchers.
Responsibilities
Figure's vision is to deploy autonomous humanoids at a global scale. Our Helix team is looking for an experienced Training Infrastructure Engineer to take our infrastructure to the next level. This role is focused on managing the training cluster, implementing distributed training algorithms, data loaders, and developer tools for AI researchers.
Responsibilities
- Design, deploy, and maintain Figure's training clusters
- Architect, optimize, and maintain scalable deep learning frameworks for training on massive robot datasets
- Work together with AI researchers to implement training of new model architectures at a large scale
- Implement distributed training, advanced parallelization strategies, and high-performance data loaders to reduce model development cycles
- Profile, identify, and eliminate training bottlenecks at the hardware and software levels to maximize Model FLOPs Utilization (MFU)
- Implement tooling for data processing, model experimentation, and continuous integration
Requirements
- Strong software engineering fundamentals
- Bachelor's or Master's degree in Computer Science, Robotics, Engineering, or a related field
- Extensive professional experience with Python and PyTorch
- Proven track record of scaling and running large-scale training experiments personally on 800+ GPUs
- Experience managing HPC clusters for deep neural network training
- Minimum of 4 years of professional, full-time experience building reliable backend systems and infrastructure
Bonus Qualifications
- Experience contributing to or maintaining open-source distributed training frameworks (Megatron-LM, DeepSpeed, TorchTitan)
- Experience managing cloud infrastructure (AWS, Azure, GCP)
- Experience with job scheduling / orchestration tools (SLURM, Kubernetes, LSF, etc.)
- Experience with configuration management tools (Ansible, Terraform, Puppet, Chef, etc.)
- Deep understanding of CUDA and hands-on experience writing custom GPU kernels to optimize training
The US base salary range for this full-time position is between $200,000 - $400,000 annually.
The pay offered for this position may vary based on several individual factors, including job-related knowledge, skills, and experience. The total compensation package may also include additional components/benefits depending on the specific role. This information will be shared if an employment offer is extended.
Show full description