README.md

🚀 Neural Networks: Zero to Hero

Welcome to my Neural Networks Training Repo! This repository captures everything I learn as I dive into the Zero to Hero course, building neural networks, language models, and even GPT from scratch.

---

📚 What’s Inside?

1️⃣ Neural Networks & Backpropagation

• Build a micro neural network framework: micrograd.
• Learn backpropagation step-by-step.
• 📝 Lecture 1 Notebook | 🔗 Micrograd Repo

---

2️⃣ Bigram Language Model

• Create a character-level bigram language model.
• Explore model training, sampling, and loss evaluation.
• 📝 Lecture 2 Notebook | 🔗 Makemore Repo

---

3️⃣ Multilayer Perceptrons (MLP)

• Implement and train an MLP language model.
• Learn hyperparameter tuning, evaluation, and avoiding overfitting.
• 📝 Lecture 3 Notebook

---

4️⃣ Advanced MLP: BatchNorm & Gradients

• Dive into activations, gradients, and BatchNorm.
• Visualize and debug your deep networks.
• 📝 Lecture 4 Notebook

---

5️⃣ Backpropagation Deep Dive

• Manually implement backpropagation for an MLP.
• Gain an intuitive understanding of gradient flow.
• 📝 Lecture 5 Notebook | 🔗 Google Colab Exercise

---

6️⃣ Building a WaveNet

• Create a hierarchical CNN inspired by WaveNet.
• Explore torch.nn and efficient model development.
• 📝 Lecture 6 Notebook

---

7️⃣ Building GPT from Scratch

• Implement a GPT following the "Attention is All You Need" paper.
• Learn the building blocks of transformers.
• 📝 Lecture 7 Notebook

---

8️⃣ Tokenizers in GPT

• Build and understand the GPT Tokenizer.
• Explore how tokenization impacts LLMs.
• 📝 Lecture 8 Notebook | 🔗 minBPE Code

---

🎯 Goals

• 📖 Master neural networks, language models, and GPT.
• 💻 Gain hands-on experience with PyTorch and deep learning tools.
• 🧠 Build intuition and confidence in machine learning.

---

🚀 How to Use

1. Clone the Repository:

   git clone https://github.com/Cluab/neural-networks-training.git
   cd neural-networks-training

2. Set Up the Environment:

   python -m venv venv
   source venv/Scripts/activate
   # Ensure the virtual environment is active before proceeding
   pip install -r requirements.txt

3. Explore the Content:

   • Navigate to the lectures folder to access notebooks and code.

4. Follow the Course:

   • Watch the YouTube Playlist for guided learning.

5. Engage with Exercises:

   • Attempt the exercises independently before reviewing the provided solutions.

---

🛠 Tools & Resources

• Languages: Python, PyTorch
• Guides:
  • PyTorch Docs
  • Micrograd Repo
  • Makemore Repo

---

📃 License

This repository follows the MIT License. See the LICENSE file for details.