Projeto.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# First example of GAN's Networks\n",
    "\n",
    "Based on this medium article: https://medium.com/@devnag/generative-adversarial-networks-gans-in-50-lines-of-code-pytorch-e81b79659e3f"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# To-do list:\n",
    "\n",
    "* () - Ver o por quê de ser tantas épocas;\n",
    "\n",
    "* () - Ver a questão da frequência de treinamento;\n",
    "\n",
    "* () - Ver a questão do detach();\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Notebook"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "There are really only 5 components to think about:\n",
    "\n",
    "R: The original, genuine data set\n",
    "\n",
    "I: The random noise that goes into the generator as a source of entropy\n",
    "\n",
    "G: The generator which tries to copy/mimic the original data set\n",
    "\n",
    "D: The discriminator which tries to tell apart G’s output from R\n",
    "\n",
    "The actual ‘training’ loop where we teach G to trick D and D to beware G.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import torch as tc\n",
    "import numpy as np\n",
    "import torch.nn as nn\n",
    "\n",
    "import torch.nn.functional as F\n",
    "import torch.optim as optim\n",
    "from torch.autograd import Variable\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Criando o <b>R</b>:\n",
    "\n",
    "Uma distribuição Gaussiana com média 4,0 e desvio-padrão 1,25."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#Criamos a função que é o dataset R, ou seja, a data que vamos copiar:\n",
    "def get_distribution_sampler(mu, sigma):\n",
    "    return lambda n: tc.Tensor(np.random.normal(mu, sigma, (1, n))) # Gaussiana"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Criando o <b>I</b>:\n",
    "\n",
    "Uma distribuição uniforme."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#Função do ruído utilizado para gerar entropia\n",
    "\n",
    "def get_generator_input_sampler():\n",
    "    return lambda m,n: tc.rand(m,n)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Criando o <b>G</b>:\n",
    "\n",
    "A nossa rede geradora.\n",
    "\n",
    "G is going to get the uniformly distributed data samples from I and somehow mimic the normally distributed samples from R.\n",
    "\n",
    "\n",
    "* O artigo utiliza ELU ao invés de RELU, estamos seguindo esta orientação apenas por simplicidade."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class Generator(nn.Module):\n",
    "    \n",
    "    def __init__(self,input_size,hidden_size,output_size):\n",
    "        #No modelo do pytorch/python vc é obrigado a herdar o init da sua classe pai\n",
    "        super(Generator,self).__init__()\n",
    "        self.map1 = nn.Linear(input_size,hidden_size)\n",
    "        self.map2 = nn.Linear(hidden_size,hidden_size)\n",
    "        self.map3 = nn.Linear(hidden_size, output_size)\n",
    "    \n",
    "    #Testar depois com relu\n",
    "    def forward(self, x):\n",
    "        x = F.elu(self.map1(x))\n",
    "        x = F.sigmoid(self.map2(x))\n",
    "        return self.map3(x)\n",
    "\n",
    "    \n",
    "    \n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Criando o <b>D</b>:\n",
    "\n",
    "Esta rede pegará samples de \"R\" (dataset real) ou de \"G\" (dataset sintético) e vai\n",
    "ter como saída valores no range de (fake,real) = (0,1).\n",
    "\n",
    "*Note que as redes tem o mesmo tamanho, mas isso não é necessário."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class Discriminator(nn.Module):\n",
    "    def __init__(self,input_size, hidden_size, output_size):\n",
    "        super(Discriminator,self).__init__()\n",
    "        self.map1 = nn.Linear(input_size,hidden_size)\n",
    "        self.map2 = nn.Linear(hidden_size,hidden_size)\n",
    "        self.map3 = nn.Linear(hidden_size, output_size)\n",
    "        \n",
    "    def forward(self,x):\n",
    "        x = F.elu(self.map1(x))\n",
    "        x = F.sigmoid(self.map2(x))\n",
    "        x = self.map3(x)\n",
    "        return(x)\n",
    "    \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Corpo do treinamento"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def train_both_nets(D,G,num_epochs,d_steps,g_steps,print_interval = 100):\n",
    "\n",
    "    use_gpu = tc.cuda.is_available()\n",
    "\n",
    "        \n",
    "\n",
    "    for epoch in range(num_epochs):\n",
    "\n",
    "        #Treina o detetive\n",
    "        for d_index in range(d_steps):\n",
    "            \n",
    "            D.zero_grad()\n",
    "            \n",
    "            # 1A - Treine o Discriminante no dataset Real\n",
    "            d_real_data = Variable(d_sampler(d_input_size))\n",
    "            d_real_decision = D(preprocess(d_real_data))\n",
    "            d_real_error = criterion(tc.clamp(d_real_decision, min=0,max=1), Variable(tc.ones(1))) #ones = true(input_size=d_input_func(d_input_size)\n",
    "            d_real_error.backward() #Computa os gradientes, mas não muda os parãmetros (não proapga?\n",
    "\n",
    "            # 1B- Treina o Discriminante no dataset Falso\n",
    "            d_gen_input = Variable(gi_sampler(minibatch_size, g_input_size))\n",
    "            d_fake_data = G(d_gen_input).detach()\n",
    "            d_fake_decision = D(preprocess(d_fake_data.t()))\n",
    "            d_fake_error = criterion(tc.clamp(d_fake_decision,min=0,max=1), Variable(tc.zeros(1))) #zeros = fake\n",
    "            d_fake_error.backward()\n",
    "            d_optimizer.step() #Atualiza os pesos da Discriminant, baseado nos gradientes de 2 erros.\n",
    "\n",
    "        #Treina o falsificador\n",
    "        for g_index in range(g_steps):\n",
    "\n",
    "            # 2. Treine falsificador com as respostas do Detetive, mas não treine D (nesses labels)\n",
    "            G.zero_grad() #sempre zere os gradientes no inicio do loop\n",
    "            # O input da rede geradora é o ruído gerador de entropia\n",
    "            gen_input = Variable(gi_sampler(minibatch_size, g_input_size))\n",
    "            g_fake_data = G(gen_input)\n",
    "            #Detetive decide sobre a criação feita pelo falsificador(input_size=d_input_func(d_input_size)\n",
    "            dg_fake_decision = D(preprocess(g_fake_data.t()))\n",
    "            g_error = criterion(tc.clamp(dg_fake_decision,min=0,max=1), Variable(tc.ones(1))) #Calcula a perda entre o dito e uma matriz de 1's\n",
    "\n",
    "            g_error.backward() # Calcula a propagação do erro\n",
    "            g_optimizer.step() # Atualiza os parâmetros do G's.\n",
    "\n",
    "        #Print do artigo\n",
    "        if epoch % print_interval == 0:\n",
    "            print(\"%s: D: %s/%s G: %s (Real: %s, Fake: %s) \" % (epoch,\n",
    "                                                                extract(d_real_error)[0],\n",
    "                                                                extract(d_fake_error)[0],\n",
    "                                                                extract(g_error)[0],\n",
    "                                                                stats(extract(d_real_data)),\n",
    "                                                                stats(extract(d_fake_data))))\n",
    "    return (D,G)\n",
    "tats(extract(d_fake_data))))\n",
    "    return (D,G)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def extract(v):\n",
    "    return v.data.storage().tolist()\n",
    "\n",
    "def stats(d):\n",
    "    return [np.mean(d), np.std(d)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def decorate_with_diffs(data, exponent):\n",
    "    mean = tc.mean(data.data, 1, keepdim=True)\n",
    "    mean_broadcast = tc.mul(tc.ones(data.size()), mean.tolist()[0][0])\n",
    "    diffs = tc.pow(data - Variable(mean_broadcast), exponent)\n",
    "    return tc.cat([data, diffs], 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Main part of the program"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Variáveis referentes às distribuições de probabilidade"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Data params\n",
    "data_mean = 4\n",
    "data_stddev = 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "### Variáveis referentes às redes generativas DETETIVE e FALSIFICADOR"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Model params\n",
    "g_input_size = 1     # Random noise dimension coming into generator, per output vector\n",
    "g_hidden_size = 50   # Generator complexity\n",
    "g_output_size = 1    # size of generated output vector\n",
    "d_input_size = 100   # Minibatch size - cardinality of distributions\n",
    "d_hidden_size = 50   # Discriminator complexity\n",
    "d_output_size = 1    # Single dimension for 'real' vs. 'fake'\n",
    "minibatch_size = d_input_size"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Variáveis referentes ao treinamento"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "d_learning_rate = 2e-4  # 2e-4\n",
    "g_learning_rate = 2e-4\n",
    "optim_betas = (0.9, 0.999)\n",
    "num_epochs = 30000 # Ver se realmente são tantas época aí\n",
    "d_steps = 1  # 'k' steps in the original GAN paper. Can put the discriminator on higher training freq than generator\n",
    "g_steps = 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Quais dados estão sendo utilizados"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# ### Uncomment only one of these\n",
    "#(name, preprocess, d_input_func) = (\"Raw data\", lambda data: data, lambda x: x)\n",
    "(name, preprocess, d_input_func) = (\"Data and variances\", lambda data: decorate_with_diffs(data, 2.0), lambda x: x * 2)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Using data [Data and variances]\n"
     ]
    }
   ],
   "source": [
    "print(\"Using data [%s]\" % (name))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Programa"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Vamos entender como funciona os samplers!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "d_sampler = get_distribution_sampler(data_mean, data_stddev)\n",
    "gi_sampler = get_generator_input_sampler()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Modelos"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#Istancia das duas redes\n",
    "G = Generator(input_size=g_input_size, hidden_size=g_hidden_size, output_size=g_output_size)\n",
    "D = Discriminator(input_size = d_input_func(d_input_size), hidden_size=d_hidden_size, output_size=d_output_size)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "criterion = nn.BCELoss()  # Binary cross entropy: http://pytorch.org/docs/nn.html#bceloss\n",
    "#Adam optimizer para redes discriminativas com parâmetros, lr= 2e-4, Beta=0.9 e gama = 0.999\n",
    "#Da teoria de otimização contínua, coloque o Beta proximo de 0.9 e vá aumentando gama até antes de divergir\n",
    "d_optimizer = optim.Adam(D.parameters(), lr=d_learning_rate, betas=optim_betas)\n",
    "g_optimizer = optim.Adam(G.parameters(), lr=g_learning_rate, betas=optim_betas)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Treinamento"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "print_interval = 200\n",
    "\n",
    "use_gpu = tc.cuda.is_available()\n",
    "\n",
    "#if use_gpu:\n",
    " #   D,G = D.cuda(), G.cuda()\n",
    "#    print(\"done\")\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/gabriel/anaconda3/envs/py35/lib/python3.5/site-packages/torch/nn/functional.py:767: UserWarning: Using a target size (torch.Size([1])) that is different to the input size (torch.Size([1, 1])) is deprecated. Please ensure they have the same size.\n",
      "  \"Please ensure they have the same size.\".format(target.size(), input.size()))\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0: D: 0.05390967056155205/0.1371496468782425 G: 2.0549092292785645 (Real: [4.0366899156570435, 1.0187636422532909], Fake: [0.38112356513738632, 0.0121997835790393]) \n",
      "200: D: 0.012576114386320114/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.9617966091632844, 0.88256202822420626], Fake: [1.8885375034809113, 0.057050503363188135]) \n",
      "400: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.0895655095577244, 1.0639598380221824], Fake: [2.0057375431060791, 0.071615221580520139]) \n",
      "600: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.0670987522602084, 1.0134667359627552], Fake: [2.1116127848625181, 0.085502138541410824]) \n",
      "800: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.8839497321844103, 0.9615737455749056], Fake: [2.2225888419151305, 0.094455737604130202]) \n",
      "1000: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.1755913305282597, 0.9328939319415952], Fake: [2.3560249066352843, 0.11425357941432109]) \n",
      "1200: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.8799513041973115, 1.0261080990563036], Fake: [2.3698163843154907, 0.11617445629376311]) \n",
      "1400: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.117895539999008, 0.90741378166994802], Fake: [2.4487072277069091, 0.12793389179007769]) \n",
      "1600: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.1271294867992401, 1.0293616621987653], Fake: [2.4550166511535645, 0.11432106191168125]) \n",
      "1800: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.8681013524532317, 1.1164454158227717], Fake: [2.4710282301902771, 0.1313018409302843]) \n",
      "2000: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.8774570739269256, 0.94362646508365666], Fake: [2.4727408862113953, 0.13702617011208429]) \n",
      "2200: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.9925193631649019, 1.0447714851401813], Fake: [2.4667923378944399, 0.11737402662309998]) \n",
      "2400: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.9693573904037476, 1.0473665031906205], Fake: [2.4404064536094667, 0.13250524283180323]) \n",
      "2600: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.9713192367553711, 1.1024848982661799], Fake: [2.4683777928352355, 0.122097319574771]) \n",
      "2800: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.7804020452499389, 0.87233218604123275], Fake: [2.4577297115325929, 0.11414583752199853]) \n",
      "3000: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.9552142268419264, 1.0971846991037995], Fake: [2.4503299760818482, 0.13881607619710451]) \n",
      "3200: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.9375347220897674, 1.0684160222635759], Fake: [2.460411636829376, 0.13787371829135928]) \n",
      "3400: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.0369030475616459, 0.96743959460167828], Fake: [2.4510037064552308, 0.12722667856247835]) \n",
      "3600: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.890211547613144, 1.0470114099504753], Fake: [2.4448315930366515, 0.12665956358570588]) \n",
      "3800: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.0023296844959262, 1.012733810874507], Fake: [2.4599954438209535, 0.12952658937342604]) \n",
      "4000: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.9849232542514801, 0.92473565404356317], Fake: [2.4701462578773499, 0.12359724120648855]) \n",
      "4200: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.1125229656696316, 0.87702270533701454], Fake: [2.4360630822181704, 0.13204041929338842]) \n",
      "4400: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.8613686442375181, 1.0139975715249798], Fake: [2.4759428572654723, 0.11662630941051082]) \n",
      "4600: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.930101855993271, 0.84978770858677821], Fake: [2.4404474306106567, 0.1332827126451063]) \n",
      "4800: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.9814838933944703, 0.93328196952972597], Fake: [2.486369128227234, 0.11784109157637745]) \n",
      "5000: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.068400852680206, 1.0081658298079643], Fake: [2.4568999600410462, 0.12498667789389541]) \n",
      "5200: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.936621837615967, 0.91571373403535761], Fake: [2.4407908511161804, 0.12863325326954669]) \n",
      "5400: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.0182149600982662, 0.84304559380568445], Fake: [2.4680690383911132, 0.12528116563661387]) \n",
      "5600: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.7295288419723511, 1.0095160848282205], Fake: [2.4557095837593077, 0.12895189351783384]) \n",
      "5800: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.8713868951797483, 0.92286865736934887], Fake: [2.4547507095336916, 0.13553661872518377]) \n",
      "6000: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.9983120834827424, 1.0343625517903308], Fake: [2.462202684879303, 0.12813590882912274]) \n",
      "6200: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.1570458948612217, 1.0355783584934188], Fake: [2.4481697273254395, 0.12875372780278507]) \n",
      "6400: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.0784414911270144, 0.88807338892088905], Fake: [2.479359951019287, 0.1240994286222807]) \n",
      "6600: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.2059517049789426, 0.93086559473442709], Fake: [2.4699078273773192, 0.13479117179502878]) \n",
      "6800: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.9673537433147432, 1.0103532979551464], Fake: [2.4440193223953246, 0.12705358800134109]) \n",
      "7000: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.0208866775035856, 1.0388491783676568], Fake: [2.488558328151703, 0.12640751400748984]) \n",
      "7200: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.0220318889617923, 1.0497462460589189], Fake: [2.4483960795402528, 0.12398755517671957]) \n",
      "7400: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.0006463122367855, 1.0326289007388079], Fake: [2.463318943977356, 0.12064680352731104]) \n",
      "7600: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.9208733344078066, 0.88044591942684036], Fake: [2.4813855957984923, 0.12924742179247209]) \n",
      "7800: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.0302494752407076, 1.0092040225391532], Fake: [2.4636417603492737, 0.12464479892543298]) \n",
      "8000: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.9954876732826232, 1.0449617880877733], Fake: [2.4457517552375792, 0.12682262631024457]) \n",
      "8200: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.9097031331062317, 0.92995697730975979], Fake: [2.4644538974761963, 0.130656455065501]) \n",
      "8400: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.2022297608852384, 1.0412584779427654], Fake: [2.4648878312110902, 0.13097485850065227]) \n",
      "8600: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.9575071656703948, 1.0365534208889091], Fake: [2.4519399666786192, 0.13426423075015151]) \n",
      "8800: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.9415629053115846, 0.96746647713110068], Fake: [2.4581676983833312, 0.13186851936255467]) \n",
      "9000: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.0097624766826634, 1.0330365336531959], Fake: [2.4683931303024291, 0.1247135236971097]) \n",
      "9200: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.8508423519134523, 1.1928329780824647], Fake: [2.4592723298072814, 0.12776257188055162]) \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "9400: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.8222414863109591, 1.0821972872836247], Fake: [2.4665086793899538, 0.13351851882875537]) \n",
      "9600: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.1051374828815463, 1.1336771564267185], Fake: [2.4462944817543031, 0.12706320044702435]) \n",
      "9800: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.0757796454429629, 0.82678367909228467], Fake: [2.4594511032104491, 0.12923275094156728]) \n",
      "10000: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.0745465314388278, 1.0573504737263109], Fake: [2.4287514805793764, 0.13075617616208707]) \n",
      "10200: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.0141352319717409, 1.0398673582364173], Fake: [2.4433907866477966, 0.1263448505472671]) \n",
      "10400: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.9172781538963317, 0.84153414672019977], Fake: [2.4635661792755128, 0.13669301383069624]) \n",
      "10600: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.8153648614883422, 0.94299520185656749], Fake: [2.4547753810882567, 0.12548470583504012]) \n",
      "10800: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.0402214550971989, 0.89837858213735644], Fake: [2.4639722275733948, 0.13191824151595791]) \n",
      "11000: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.8215816295146943, 0.93009639994103477], Fake: [2.4719005417823792, 0.11803544462883707]) \n",
      "11200: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.0298459219932559, 1.1201556501944905], Fake: [2.4547367548942565, 0.12562132396540363]) \n",
      "11400: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.913429309129715, 0.93740463232044369], Fake: [2.4540572810173034, 0.12683246666711728]) \n",
      "11600: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.0806713163852688, 1.0326626438049038], Fake: [2.4667449021339416, 0.12762882754103597]) \n",
      "11800: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.1258417302370072, 0.9808269987488849], Fake: [2.4657944011688233, 0.12107717724032828]) \n",
      "12000: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.9372318255901337, 0.93634113823633125], Fake: [2.4514476823806763, 0.13027591910714334]) \n",
      "12200: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.9572727775573728, 0.94189545964004173], Fake: [2.4797687053680422, 0.12753561721009102]) \n",
      "12400: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.978871680498123, 1.0370354058696594], Fake: [2.47588919878006, 0.13186380494779001]) \n",
      "12600: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.0436163365840914, 1.0245249280943045], Fake: [2.4353615856170654, 0.12591904869248327]) \n",
      "12800: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.0135075783729555, 0.97386408695614435], Fake: [2.4636822724342347, 0.12457415792496103]) \n",
      "13000: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.1205065321922305, 0.89657475855187985], Fake: [2.4640267443656922, 0.12503564713460627]) \n",
      "13200: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.248316460847855, 1.0038788543531825], Fake: [2.4476829242706297, 0.12610180236320956]) \n",
      "13400: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.0692183530330661, 1.0564315281490337], Fake: [2.4521469950675963, 0.13330157750144028]) \n",
      "13600: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.8726318538188935, 0.97850517825203809], Fake: [2.4721706485748292, 0.11847873706952912]) \n",
      "13800: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.2033767569065095, 0.98782307436882355], Fake: [2.4712247633934021, 0.12415515477366262]) \n",
      "14000: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.9491052138805389, 1.1131023873377384], Fake: [2.4599035382270813, 0.13416184575196674]) \n",
      "14200: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.9745076858997344, 0.95731704769822501], Fake: [2.4689557123184205, 0.13194310319421798]) \n",
      "14400: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.0880813777446745, 0.99886187818877858], Fake: [2.4627363157272337, 0.125900475412355]) \n",
      "14600: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.9051725459098816, 0.89631154219177955], Fake: [2.4560135364532472, 0.13181806562473297]) \n",
      "14800: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.8992104947566988, 0.99197531802729166], Fake: [2.44155086517334, 0.12653922503628118]) \n",
      "15000: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.9446908807754517, 0.86459845843978411], Fake: [2.4724779510498047, 0.12128128874214053]) \n",
      "15200: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.9398402202129366, 0.87389114148255387], Fake: [2.4569054603576661, 0.11898415075613906]) \n",
      "15400: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.9288243710994721, 0.91933895269615906], Fake: [2.4600582766532897, 0.13442395586093803]) \n",
      "15600: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.0071231579780582, 0.91802211011314583], Fake: [2.4518449163436888, 0.1274463328197355]) \n",
      "15800: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.9757142746448517, 1.0593804203743473], Fake: [2.4674229741096498, 0.12941601000083208]) \n",
      "16000: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.988950593471527, 0.95535620748251937], Fake: [2.4767958259582521, 0.12909477506347947]) \n",
      "16200: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.1494323015213013, 0.98746082408728975], Fake: [2.4594814634323119, 0.12274105105200965]) \n",
      "16400: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.0331176573038103, 0.9418290235998541], Fake: [2.4654260730743407, 0.11828516803404367]) \n",
      "16600: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.9429631650447847, 1.0277912263877407], Fake: [2.4420841336250305, 0.12224546272763086]) \n",
      "16800: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.9715707468986512, 1.0951715347037889], Fake: [2.4587235331535338, 0.12635312035142895]) \n",
      "17000: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.981491037607193, 0.93295727451204857], Fake: [2.4404931545257567, 0.12434336318354887]) \n",
      "17200: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.057705236673355, 1.0857883202645311], Fake: [2.4623848223686218, 0.126038344044935]) \n",
      "17400: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.8927925145626068, 1.0792480757620049], Fake: [2.4595179033279417, 0.12687314983927608]) \n",
      "17600: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.9861739134788512, 1.0014375010253462], Fake: [2.4481130981445314, 0.13220966135907863]) \n",
      "17800: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.0393413519859314, 0.89438927266142743], Fake: [2.4593567323684691, 0.13145059232438261]) \n",
      "18000: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.9329778337478638, 0.9913430130097296], Fake: [2.4596712160110474, 0.13521194500654007]) \n",
      "18200: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.0378558301925658, 0.90915040889259313], Fake: [2.4568335342407228, 0.13236691735957087]) \n",
      "18400: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.0210020422935484, 0.94368872099023504], Fake: [2.4451443648338316, 0.12718967383524887]) \n",
      "18600: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.2049214446544649, 0.95019620521346171], Fake: [2.4573747181892394, 0.13156539501806835]) \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "18800: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.0789195454120639, 0.89288775450831437], Fake: [2.4602111148834229, 0.12163163747831909]) \n",
      "19000: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.9242075645923613, 0.95835705191235632], Fake: [2.4560175275802614, 0.12534866552335514]) \n",
      "19200: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.9301237285137178, 0.99231781899445359], Fake: [2.4757011055946352, 0.11895233447998119]) \n",
      "19400: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.0061802864074707, 1.0465876831770151], Fake: [2.454839165210724, 0.1288185463080167]) \n",
      "19600: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.0751066064834598, 1.0476726836377503], Fake: [2.4617360687255858, 0.12182696922923031]) \n",
      "19800: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.9532866311073303, 1.0524291228314135], Fake: [2.4357474637031555, 0.12493945176085088]) \n",
      "20000: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.960196579694748, 0.99817342013143517], Fake: [2.449796049594879, 0.11216880251859494]) \n",
      "20200: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.9597785878181457, 0.96292545372740801], Fake: [2.4558814692497255, 0.13792441886710136]) \n",
      "20400: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.0597661890089514, 1.1557299775266656], Fake: [2.4521669006347655, 0.12415094532369399]) \n",
      "20600: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.0567223203182223, 0.9755328881376204], Fake: [2.4395998382568358, 0.12904854635897273]) \n",
      "20800: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.8593128490447999, 0.88599465651812737], Fake: [2.4544274687767027, 0.12777702153624104]) \n",
      "21000: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.9387805175781252, 0.93763915739186021], Fake: [2.4673789715766907, 0.12045203636933519]) \n",
      "21200: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.7894049572944639, 1.001636388155126], Fake: [2.466751763820648, 0.12857119190935409]) \n",
      "21400: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.0365134394168853, 0.98942464983146805], Fake: [2.4660584640502932, 0.12453516545593105]) \n",
      "21600: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.816843811273575, 1.0715030054043524], Fake: [2.446326582431793, 0.11783907412031254]) \n",
      "21800: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.1860024213790892, 1.0297006685816488], Fake: [2.4666591882705688, 0.12767749581900747]) \n",
      "22000: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.2783848547935488, 0.94636632375344731], Fake: [2.4740037584304808, 0.12438739345373445]) \n",
      "22200: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.0354716610908508, 0.93977738110935938], Fake: [2.4455497694015502, 0.1279232130888206]) \n",
      "22400: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.925353968143463, 0.99822835700285817], Fake: [2.4735613989830019, 0.13198235303491318]) \n",
      "22600: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.0437904071807864, 1.0388497464478446], Fake: [2.4387213492393496, 0.12868391805567794]) \n",
      "22800: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.9513743269443511, 0.93536671253919645], Fake: [2.4431488752365111, 0.12938932333597741]) \n",
      "23000: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.0437911570072176, 1.0202642834389788], Fake: [2.465657172203064, 0.13542521017909481]) \n",
      "23200: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.0304647207260134, 1.0438454584293178], Fake: [2.4457250380516053, 0.13512117068537435]) \n",
      "23400: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.1191828608512875, 1.0156391624675756], Fake: [2.4374001741409304, 0.12103732753122901]) \n",
      "23600: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.0802345013618471, 1.044092397211817], Fake: [2.4752753829956053, 0.12259731333705774]) \n",
      "23800: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.0589438402652744, 0.90765195118613984], Fake: [2.4744743490219117, 0.12262844783586772]) \n",
      "24000: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.9961995637416838, 1.0000133303371634], Fake: [2.4579791569709779, 0.12821068985417369]) \n",
      "24200: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.8796954107284547, 0.96250115257933255], Fake: [2.4576643919944763, 0.12395434301966089]) \n",
      "24400: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.0745752775669102, 0.89430363036930549], Fake: [2.4581420159339906, 0.12879122922702735]) \n",
      "24600: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.0689044952392575, 0.99164102655779318], Fake: [2.4613679790496827, 0.12287578401306908]) \n",
      "24800: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.0265691506862638, 0.99107096011699947], Fake: [2.4487380099296572, 0.13467983777492126]) \n",
      "25000: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.0758254361152648, 1.0185324764263255], Fake: [2.464938848018646, 0.13044098099939827]) \n",
      "25200: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.8558827447891235, 0.98293147838285444], Fake: [2.4625040078163147, 0.13305438283728005]) \n",
      "25400: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.8238904225826262, 1.1096414447492267], Fake: [2.47765371799469, 0.11518237372035481]) \n",
      "25600: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.9127916216850283, 0.88297918401092379], Fake: [2.4600192213058474, 0.12833421430867634]) \n",
      "25800: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.1382135963439941, 1.035708630335137], Fake: [2.4482121920585631, 0.13440032898884202]) \n",
      "26000: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.9462456321716308, 0.8967748189329432], Fake: [2.4584045696258543, 0.13698358642845992]) \n",
      "26200: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.0773837828636168, 0.99495947694126796], Fake: [2.465447919368744, 0.13786363869306786]) \n",
      "26400: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.163632389307022, 0.87983595318331653], Fake: [2.4484069252014162, 0.12763894343661725]) \n",
      "26600: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.9232277643680571, 0.97969590547259822], Fake: [2.4533039546012878, 0.13769980615342259]) \n",
      "26800: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.982058925628662, 0.9867253660489621], Fake: [2.4658469510078431, 0.12574843011150869]) \n",
      "27000: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.1910349428653717, 0.91095461429748503], Fake: [2.4555366492271422, 0.13311996217510663]) \n",
      "27200: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.9131101846694945, 0.87815506343360128], Fake: [2.4639040637016296, 0.12379787809658906]) \n",
      "27400: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.8485570406913756, 1.018651202682789], Fake: [2.4562089252471924, 0.11773171461830877]) \n",
      "27600: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.8884675121307373, 0.96423204238321569], Fake: [2.4535796451568603, 0.12878267088965567]) \n",
      "27800: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.1774339985847471, 0.9658648311539666], Fake: [2.4557949972152708, 0.124713502692229]) \n",
      "28000: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.9423687839508057, 1.1161857767736552], Fake: [2.475454981327057, 0.13141559207162051]) \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "28200: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.8464529645442962, 0.97004526445354755], Fake: [2.4385418915748596, 0.12083711460775341]) \n",
      "28400: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.1675326001644137, 1.0427119434222172], Fake: [2.4712990736961364, 0.12340838615604922]) \n",
      "28600: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.1578790473937985, 0.92090345940334983], Fake: [2.4790980982780457, 0.12050110072731238]) \n",
      "28800: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.2964280200004579, 1.1383489412357672], Fake: [2.4466008687019349, 0.12158073094076159]) \n",
      "29000: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.9353268301486968, 1.063629898288184], Fake: [2.4760322380065918, 0.11785899482886121]) \n",
      "29200: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.9920670342445375, 0.93796633475394864], Fake: [2.4757920002937315, 0.12717889026522333]) \n",
      "29400: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.1604032373428348, 0.95625293098955], Fake: [2.4641549611091613, 0.13080766120622281]) \n",
      "29600: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [4.0783550763130192, 0.86757333563102301], Fake: [2.4689754700660704, 0.12302263602658173]) \n",
      "29800: D: -1.000088900582341e-12/-1.000088900582341e-12 G: 27.63102149963379 (Real: [3.8373579943180083, 0.88508053075143178], Fake: [2.4740148186683655, 0.13884070972845039]) \n"
     ]
    }
   ],
   "source": [
    "D,G = train_both_nets(D = D, G = G, num_epochs = num_epochs, d_steps = 1, g_steps = 1,print_interval=print_interval)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Solution"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " Entendendo o código\n",
    "\n",
    "Na parte '1A' do treino do detetive, jogamos (foward step) para o detetive dados reais e dados fakes, afim de achar os 2 erros com o criterion de custo. O método backward() calcula os gradientes no grafo de computação e vai ser guardado para atualizar os pesos da rede, feito no método d_optimizer.step(). \n",
    "\n",
    "*OBS: Note que G é utilizado apenas para fazer predict ali, por tanto não é treinado.\n",
    "\n",
    "Já na parte '2A', que é o treinamento do falsificador, criamos uma tela falsa (g_fake_data) e damos para o detetive decidir o que acha da falsificação (dg_fake_decision) para só depois calcularmos o erro do critério de custo (d_fake_error). Após isso calculamos os gradientes do grafo de computação e utilizamos para atualizar os pesos no g_optimizer.step().\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}