8 Advanced parallelization - Deep Learning with JAX

Using easy-to-revise parallelism with xmap() · Compiling and automatically partitioning functions with pjit() · Using tensor sharding to achieve parallelization with XLA · Running code in multi-host configurations

8 Advanced parallelization - Deep Learning with JAX

Frontiers Tensor Processing Primitives: A Programming

Vectorize and Parallelize RL Environments with JAX: Q-learning at

Using Cloud TPU Multislice to scale AI workloads

Running a deep learning workload with JAX on multinode multi-GPU

SWARM Parallelism: Training Large Models Can Be Surprisingly

Build a Transformer in JAX from scratch: how to write and train

Model Parallelism

11.7. The Transformer Architecture — Dive into Deep Learning 1.0.3

Using JAX to accelerate our research - Google DeepMind

Compiler Technologies in Deep Learning Co-Design: A Survey

Tools for infrastructure for MLOps