Permanent positions
We are always searching for strong applications for permanent Researcher (CR CNRS, CR Inria, Maître de Conférences) positions within the team. Please contact us if you are interested to apply.
Job offers
Efficient Training of Neural Networks (Internship)
Supervised by Julia Gusak, Lionel Eyraud-Dubois and Olivier Beaumont.
The goal of this project is to study and implement different techniques for efficient multi-GPU training. Firstly, you’ll implement baselines using different activation checkpointing techniques from Rockmate and pipelining techniques from PyTorch and DeepSpeed. Then you’ll develop and implement new approaches for combining pipelining and checkpointing. You will compare them in terms of memory footprint / computational time trade-off, and contribute to the software package. You’ll perform experiments with modern neural networks, including GPT-like models and Neural Operators. You’ll analyze the performance of models using NVIDIA and PyTorch profiling tools.