GitHub - pavanpreet-gandhi/reproduce-gpt2: Following along Andrej Karpathy reproducing GPT-2 from scratch.

pavanpreet-gandhi / reproduce-gpt2 Public

Following along Andrej Karpathy reproducing GPT-2 from scratch.

Notifications

Name		Name	Last commit message	Last commit date
Latest commit History 13 Commits
data		data
.gitignore		.gitignore
1_huggingface_transformers.ipynb		1_huggingface_transformers.ipynb
2_gpt2_from_pretrained.ipynb		2_gpt2_from_pretrained.ipynb
3_sampling_from_models.ipynb		3_sampling_from_models.ipynb
4_overfitting_single_batch.ipynb		4_overfitting_single_batch.ipynb
5_simple_training_loop.ipynb		5_simple_training_loop.ipynb
6_timing_baseline.ipynb		6_timing_baseline.ipynb
7_computational_optimizations.ipynb		7_computational_optimizations.ipynb
8_correct_hyperparameters.ipynb		8_correct_hyperparameters.ipynb
9_gradient_accumulation.ipynb		9_gradient_accumulation.ipynb
dataloader.py		dataloader.py
gpt2.py		gpt2.py
train.py		train.py