Part 6: Training a GPT-like model with DDP (code walkthrough)

Published 2022-09-20

Download video MP4 360p

Recommendations

1:56:20

Let's build GPT: from scratch, in code, spelled out.
35:53

Official PyTorch Documentary: Powering the AI Revolution
18:31

Testing Frontier LLMs (GPT4) on ARC-AGI
1:12:53

Distributed Training with PyTorch: complete tutorial with cloud infrastructure and code
24:19

A friendly introduction to distributed training (ML Tech Talks)
32:31

How Fully Sharded Data Parallel (FSDP) works?
16:13

Lightning Talk: Triton Compiler - Thomas Raoux, OpenAI
41:55

I learned to code from scratch in 1 year. Here's how.
15:04

How I'd Learn AI (If I Had to Start Over)
30:54

Diffusion models from scratch in PyTorch
39:13

Introduction to PyTorch Tensors
22:07

Introducing ExecuTorch from PyTorch Edge: On-Device AI... - Mergen Nachin & Orion Reblitz-Richardson
05:35

Training on multiple GPUs and multi-node training with PyTorch DistributedDataParallel
09:41

Learn To Code Like a GENIUS and Not Waste Time
1:42:28

CHATGPT DOESN'T REASON! (Top scientist bombshell)
17:07

LoRA explained (and a bit about precision and quantization)

Similar videos

10:14

Part 3: Multi-GPU training with DDP (code walkthrough)
01:57

Part 1: Welcome to the Distributed Data Parallel (DDP) Tutorial Series
09:09

Part 5: Multinode DDP Training with Torchrun (code walkthrough)
11:07

Part 4: Multi-GPU DDP Training with Torchrun (code walkthrough)
03:47

PyTorch vs TensorFlow | Ishan Misra and Lex Fridman
27:11

Data Parallelism Using PyTorch DDP | NVAITC Webinar
4:01:26

Let's reproduce GPT-2 (124M)
59:22

minGPT code walkthrough
14:20

How To Train Large Language Models LLM like GPT 4 on PyTorch 2.0 | Distributed Model Training on GPU
37:06

Unlocking PyTorch Through Prompt Engineering, Part 1
00:50

karpathy/minGPT - Gource visualisation
46:16

How to Train a Model with Pytorch
1:08:22

Distributed Data Parallel Model Training in PyTorch
1:01:44

3% of Google code is by a LM, Data Pruning, Prompt Tuning... Trends in AI — August 2022
07:36

PyTorch Distributed: Towards Large Scale Training
More results