08_MoCo.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Welcome to your 8th assignment!\n",
    "\n",
    "Please submit your solution of this notebook in the Whiteboard at the corresponding Assignment entry. We need you to upload the .ipynb-file and the exported .pdf of this notebook.\n",
    "\n",
    "If you have any questions, ask them in either in the tutorials or in the \"Mattermost\" channel: https://mattermost.imp.fu-berlin.de/biorobotics/channels/ssl_ws_2324\n",
    "\n",
    "\n",
    "In this assignment, we want you to dive into MoCo.\n",
    "\n",
    "Have a nice Christmas Break! :)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Slide Review\n",
    "\n",
    "[Google Form](https://forms.gle/6BW1eHazybumpcf49) for the slide review of this week. Please take a minute to scroll over the slides again and improve your lecture.\n",
    "\n",
    "Please make sure to only choose your top 5 slides per lecture!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Please state both names of your group members here:\n",
    "Authors: Jane and John Doe"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Assignment 8: Momentum Contrast\n",
    "\n",
    "Reading List: <br>\n",
    "[MoCo v1 Paper](https://arxiv.org/abs/1911.057226) <br>\n",
    "[SimCLR Paper](https://arxiv.org/abs/2002.05709) <br>\n",
    "[MoCo v2 Paper](https://arxiv.org/abs/2003.04297) <br>\n",
    "\n",
    "## Ex. 8.1 MoCo v2\n",
    "\n",
    "To make it feasible to train a more recent model on small machines, we want you to implement MoCo v2, as it minimizes the memory usage compared to SimCLR. Key parts are the queue, momentum update and the handling of the affinity matrix.\n",
    "\n",
    "Re-build the MoCo v2 model architecture. **(RESULT)**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Ex. 8.2 Training on CIFAR10\n",
    "\n",
    "We want you to train the MoCo v2 model on [CIFAR10](https://www.cs.toronto.edu/~kriz/cifar.html). **(RESULT)**\n",
    "\n",
    "You can download and preprocess the dataset using PyTorch similar to MNNIST. CIFAR10 consists of colored 3x32x32 images and 10 classes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Ex. 8.3 Evaluation through Linear Probing\n",
    "\n",
    "Like in the assignments before, evaluate the quality of your learned representations using linear probing, aka. a linear classifier on your frozen pretrained model. **(RESULT)**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Code here"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}