notebooks/intro-pairwise.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction using pairwise-comparison data\n",
    "\n",
    "This notebook provides a gentle introduction to the `choix` library.\n",
    "We consider the case of pairwise-comparison outcomes between items from some set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import choix\n",
    "import networkx as nx\n",
    "import numpy as np\n",
    "\n",
    "%matplotlib inline\n",
    "np.set_printoptions(precision=3, suppress=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In `choix`, items are represented by $n$ consecutive integers $\\{0, \\ldots, n-1 \\}$.\n",
    "The event \"item $i$ wins over item $j$\" is represented by the Python tuple `(i, j)`.\n",
    "Note that the winning item always comes *first* in the tuple.\n",
    "\n",
    "We start by defining a small dataset of comparison outcomes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "n_items = 5\n",
    "data = [\n",
    "    (1, 0), (0, 4), (3, 1),\n",
    "    (0, 2), (2, 4), (4, 3),\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This dataset can be visually represented by using a graph:\n",
    "\n",
    "- each node is an item, and\n",
    "- there is an edge from node $i$ to node $j$ for every observation \"$i$ wins over $j$\"."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeIAAAFBCAYAAACrYazjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XlcVOX+B/DPiCgzCoqi5a6AoCmICrK4pGYqRCQmWWbi\nen8301yiTG3RboEVqbRo2SJ6b6XXTLPSdiNFlsGFRUVWBc0EBQFlEIHz+8Oc6xFQlpl5Zvm8X69e\nNYczz/nWrfvhe85znkchSZIEIiIiEqKF6AKIiIgsGYOYiIhIIAYxERGRQAxiIiIigRjEREREAjGI\niYiIBGIQExERCcQgJiIiEohBTEREJBCDmIiISCAGMRERkUAMYiIiIoEYxERERAIxiImIiARiEBMR\nEQnEICYiIhKIQUxERCQQg5iIiEggBjEREZFADGIiIiKBGMREREQCMYiJiIgEYhATEREJxCAmIiIS\niEFMREQkEIOYiIhIIAYxERGRQAxiIiIigRjEREREAjGIiYiIBGIQExERCcQgJiIiEohBTEREJBCD\nmIiISCAGMRERkUAMYiIiIoEYxERERAIxiImIiARiEBMREQnEICYiIhKIQUxERCQQg5iIiEggBjER\nEZFADGIiIiKBGMREREQCMYiJiIgEYhATEREJxCAmIiISiEFMREQkEIOYiIhIoJaiCyAiqktBQQG2\nRkcjIyUFV0pK0LZdO7i4uyN01ix06tRJdHlEOqOQJEkSXQQR0U1qtRpRERH4ft8+TAbgVVEBWwBl\nABKVSuySJDzk749Fy5fDy8tLcLVEzccgJiKjsWnjRrwaFoZlGg1CJQn2dZxTDCBaocBbSiVWR0bi\nH08/begyiXSKQUxERmHTxo14MywMP5aXw7kB52cBmKBSYRnDmEwcg5iIhFOr1QgaPRoHbgvhpwD8\nAqAcgAOA2QBW3vLzLAAjVSp8GxMDT09PwxVMpEOcNU1EwkVFRGCZRlOrE14OIBdACYB9AN4D8OMt\nP3cG8IJGg6iICMMUSqQH7IiJSKiCggK49uqFnIqKOp8J33QKwDgA3wAYcsvxIgBONjbIyMvjbGoy\nSeyIiUiordHRCAbqDeFnALQBMBA3bksPue3nHQAEKxTYGh2ttxqJ9IlBTETCFBUV4bd9+zCsoqLe\ncz4AcAU3nhW/BEBdxzleGg0yUlP1UySRnnFBDyIyCEmSkJubi9jYWBw8eBCxsbE4fvw42gJ48i7f\nVQC4H0AIgC8B3P72sC2AsuJi3RdNZAAMYiLSi6qqKiQnJ2tD9+DBgzh//nyt86pxY7GOBo0JQFXH\n8TIAtvZ3esJMZLwYxESkE2VlZUhISMDBgwdx8OBBxMfH4+rVq3f9XgWAGAD/vO14IYDfAAQCUAL4\nGcCOv/98O7VSiQFubs2qn0gUzpomoiY5d+6c7DbzsWPHUFNT06gxFAoF+vfvj7xTp5BXXS2bsHUR\nwBQAKQAkAH0BvAzg4dvG4KxpMnXsiInormpqanDixAlt8B48eBCnT59u9DhKpRI+Pj4YPnw4RowY\nAR8fH7Rr1w7TJ0/Glt27sfiWvsABwO8NGHOLQoHAgACGMJksdsREVEtFRQXUarU2dA8dOoTLly83\nepx77rlHG7ojRoyAh4cHrK2ta51X38pad8OVtcgcMIiJCBcvXkRsbKy2401KSsL169cbPU6/fv0w\nYsQIbfg6OTlBoVA06Ltca5osFW9NE1kYSZKQnZ2t7XZjY2ORnp7e6HGsra3h5eWlDV0/Pz84ODg0\nua6bYToyLAwvaDSYWc/uS0UANisUiOTuS2Qm2BETmbnr16/j6NGjsue7BQUFjR6nffv22tAdPnw4\nPD09oVQqdV5vUlISoiIi8N3evQhWKOCl0Wj3I1Yrldhx/Tq6d+2Kf+/cydvRZBYYxERmpqSkBPHx\n8drQTUhIgEajafQ4ffr0kT3f7d+/P1q0MNxifIWFhdgaHY2M1FSUFRfD1t4eLm5ueCQ4GN7e3khK\nSkKfPn0MVg+RvjCIiczItGnTsG3bNjT2P+sWLVrAw8ND2+0OHz4c3bp101OVzbdixQpcvnwZGzZs\nEF0KUbMxiInMRGlpKZ566ins2bPnrue2adMGvr6+2o7X29sbtra2BqhSNwoKCtCvXz8cP34cXbp0\nEV0OUbMwiIlM1NmzZ2ULamRkZKBHjx51Trzq0qWL9hbz8OHDMWjQILRsadpzNRctWoRWrVrh7bff\nFl0KUbMwiIlMQE1NDY4fPy5bt/nq1auyyVNDhgxBYWEhevTogQEDBsheI+rdu3eDXyMyFfn5+Rg0\naBAyMzPRsWNH0eUQNRmDmMgIaTQa2YIacXFx6NSpkyxcXVxc6gzX4uJi2FvIBghz585F9+7dsWrV\nKtGlEDUZg5jICBQWFspuM6ekpMDNzU32ju4999wjukyjk5mZCT8/P+Tk5JjUM26iWzGIiQxMkiRk\nZmbK3uu9cOECfH19tR3vsGHDoFLVteEf3e6JJ57AkCFD8Pzzz4suhahJGMREelZZWYmjR4/Knu8q\nlUrZO7oDBgyAlZWV6FJNUkpKCiZMmICcnBy9LDBCpG8MYiIdKykpQVxcnLbbTUpKgrOzs2zWco8e\nPUSXaVaCgoIwceJEzJ8/X3QpRI3GICZqpry8PNm6zTk5ObI1mG9u9Uf6Ex8fj8cffxyZmZl17u5E\nZMwYxESNUF1djdTUVNnz3crKSlm3O3jwYIaBAA888ABmzJiB0NBQ0aUQNQqDmOgOrl69isTERG23\nGxcXh65du8qe7zZmqz/Sn19//RXPPPMMjh8/zuftZFIYxES3+Ouvv2T78h4/fhyDBg3Shm5zt/oj\n/ZEkCb6+vggLC8OUKVNEl0PUYAxisliSJCE9PV32/u6lS5fg5+en7Xj1tdUf6ceePXuwatUqHD58\nmHcpyGQwiMliXLt2DYcPH9aGbmxsLOzs7IRu9Ue6VVNTAw8PD7z55pvw9/cXXQ5RgzCIyWwVFxfj\n0KFD2klVR48ehaurq8ls9UdN8+WXX2LDhg04cOCA6FKIGoRBTGZBkiScPn1atmhGXl4evL29TXar\nP2qa6upq9OvXD59++ilGjRoluhyiu2IQk0mqqqpCcnKy7PmuJElmt9UfNc0nn3yCr776Cj/88IPo\nUojuikFMJuHKlSuIj4/Xhm5CQgJ69uwpe75rjlv9UdNUVlbCyckJu3btgqenp+hyiO6IQUxG6c8/\n/5QtmnHq1CkMHjxY2+36+fmhQ4cOosskIxYVFYU//vgDO3fuFF0K0R0xiEm4mpoanDx5UrZMZGlp\nqXZC1YgRIzB06FC0bt1adKlkQsrLy9GnTx/s378f9913n+hyiOrFICaDq6iogFqt1na8hw4dQocO\nHWTPd11dXfkaETVbeHg40tPTsXXrVtGlENWLQUx6d/HiRdlrRMnJyRgwYIDsNaJ7771XdJlkhkpK\nSuDk5ITExEQ4OjqKLoeoTgxi0ilJkpCdnS17jejPP/+Ej4+PtuMdNmwY2rRpI7pUshArV65EUVER\nNm7cKLoUojoxiKlZrl+/jmPHjsme71pbW2u73REjRsDNzY2L8JMwhYWFcHV1RVpaGrp27Sq6HKJa\nGMTUKKWlpYiLi9N2u2q1Go6OjrLXiHr27Cm6TCKZxYsXo2XLloiMjBRdClEtDGK6o/z8fNlrRFlZ\nWfD09NR2vL6+vmjfvr3oMonu6OzZs3B3d0dmZiY6duwouhwiGQYxaVVXV+P48eOy57sajUbW7Q4e\nPBitWrUSXSpRo82bNw9du3bF6tWrRZdCJMMgtmDl5eVITEzUhm58fDw6d+4se77bt29frlZFZiEr\nKwu+vr7Izs6GnZ2d6HKItBjEFqSgoEC2NnNqairc3d21oevn54fOnTuLLpNIb6ZNmwYPDw+88MIL\noksh0mIQ60hBQQG2RkcjIyUFV0pK0LZdO7i4uyN01ix06tTJ4PVIkoSMjAzZbebCwkL4+vpqO14v\nLy+oVCqD10YkSmpqKsaPH4+cnBwolUrR5RABYBA3m1qtRlREBL7ftw+TAXhVVMAWQBmARKUSuyQJ\nD/n7Y9Hy5fDy8tJbHZWVlThy5IjsNaI2bdrIbjMPGDCAq1WRxXvkkUcwfvx4PPPMM6JLIQLAIG6W\nTRs34tWwMCzTaBAqSbCv45xiANEKBd5SKrE6MhL/ePppnVz78uXLOHTokLbbPXz4MPr27StbJrJ7\n9+46uRaROUlISMBjjz2GrKwsWFtbiy6HiEHcVJs2bsSbYWH4sbwczg04PwvABJUKy5oQxpIkIS8v\nT9bt5ubmYtiwYdpu18fHhxNQiBpo3LhxmD59OmbOnCm6FCIGcVOo1WoEjR6NA3WEcCYAdwAhAG5f\nZj4LwEiVCt/GxNxxj9Tq6mqkpKTInu9WVVXJul0PDw/+Nk/URL/99hvmz5+P48ePc9U3Eo5B3ATT\nJ0+G5+7dWFzHP7oJACoA9ELtIAaAdQoFjgQH49+37JF69epVJCQkaDvehIQEdOvWTfZ819HRka8R\nEemIJEnw8/PD0qVLERISIrocsnAM4kYqKCiAa69eyKmoqPVMeBuA3QDuw43ut64gLgLg1Lo1Ij/4\nAGlpaTh48CBOnDgBDw8Pbcfr5+fH1X+I9Ozbb7/FK6+8giNHjvCXXBKKQdxIkW+9hROvvorPKipk\nx0sBeAHYD+BjANmoO4gB4AkAKf37Y0ZoKIYPHw5PT0/Y2Njos2wiuo0kSfDw8EBERAQCAgJEl0MW\nrKXoAkxNRkoKht0WwgDwCoB5ABqyt8soAHaenli2bJmOqyOihlIoFFi+fDneeOMN+Pv7sysmYfhS\naSNdKSmB7W3HjgH4BcDiBo5hC6CsuFindRFR44WEhKCgoAB//PGH6FLIgjGIG6ltu3You+1YDIAz\nAHoC6AIgEsBXAOqbF10GwNa+rreOiciQrKys8OKLLyI8PFx0KWTBGMSN5OLujsTbnuf+H248Ez4G\nIBnAPwEEAvipnjHUSiVc3Nz0WSYRNdBTTz2FkydPQq1Wiy6FLBQnazXSnWZN37Qa9U/WKgLgZGOD\njLw8IWtQE1Ft7777Ln7//Xd8/fXXokshC8SOuJE6d+6Mh/z9seUOEzteRf0zpqMVCgQGBDCEiYzI\n3LlzcejQIRw/flx0KWSBGMRNsGj5crypVCKrkd/LArBakjBi/Hh9lEVETaRSqbBo0SKsWbNGdClk\ngXhruomastb0SAAXALRq3Rqff/45Hn30Uf0WSUQNVlJSAicnJyQmJsLR0VF0OWRBrFatWrVKdBGm\naKiXF5QdOmDG/v2wqqpCPwB17W5aBOB9ANP//msJN9aS3rFjBzp37qzXrRGJqOFsbGxQWlqKn3/+\nGYGBgaLLIQvCjriZkpKSEBURge/27kWwQgEvjUa7H3EMgG8AKAD4jh2L3377rdb3X3rpJbz22mtc\nTIDICBQWFsLV1RVpaWno2rUhy/MQNR+DWEcKCwuxNToaGampKCsuxqmsLBxNT8fNf7hubm5YsmQJ\n5s2bh+rqatl3586di40bN6JlSy50RiTakiVL0KJFC7zzzjuiSyELwSDWk9OnT8PJyQk1NTXaYz//\n/DMqKysREhKC8vJy2flBQUH48ssvoVKpDF0qEd3i7NmzcHd3R2ZmJjdfIYPgrGk96d27NyZPniw7\ntnbtWgQEBOC3336r9R/4nj178OCDD6KoqMiQZRLRbbp3744pU6YgKipKdClkIdgR61FcXBz8/Pxk\nx06cOIH+/fvj1KlTmDBhAs6cOSP7ef/+/fHjjz+iR48ehiyViG6RlZUFHx8f5OTkwM7OTnQ5ZObY\nEeuRr68vvL29ZcfWr18PAHB1dcWhQ4fgdttSlydPnoSfnx8XFiASyNnZGRMmTMDGjRtFl0IWgB2x\nnv33v//F1KlTtZ9tbGyQn58PBwcHAMDly5cxadIkxMTEyL43Z84cfPLJJwatlYj+Jy0tDQ8++CBy\ncnKgVNb1ciKRbrAj1rPJkyejZ8+e2s8VFRX48MMPtZ/bt2+PH374Qba4R+vWrbFw4UKD1klEcgMH\nDoS3tzc+/fRT0aWQmWMQ61nLli3x7LPPyo598MEHuHbtmvazjY0Ntm/fjvnz58Pd3R3r16/HhAkT\nEB8fb+hyiegWK1aswNtvv43KykrRpZAZ461pAygpKUH37t1x5coV7bHo6GiEhobKzpMkCWVlZbCz\ns8O+ffsQGhqKzZs346GHHjJ0yUT0twcffBDTpk3DrFmzRJdCZoodsQG0a9cOc+bMkR1bt24dbv8d\nSKFQaGdo+vv749tvv8XcuXOxefNmg9VKRHIrVqzAmjVrai3EQ6QrDGIDefbZZ2XLWCYnJ2P//v13\n/I63tzd+//13vPbaawgPD68V3ESkf6NHj0bHjh2xc+dO0aWQmWIQG4ijoyOCg4Nlx9auXXvX77m6\nuiI2Nhbbt2/Hs88+y9/KiQxMoVBgxYoV/GWY9IZBbEBLly6Vff7+++9x6tSpu36va9eu+OOPP5CW\nloYnnnhCNtGLiPTvoYcegiRJ2Lt3r+hSyAwxiA3Iz8+v1raHNxf4uJt27dph3759kCQJ/v7+KCkp\n0UeJRFSHm13xG2+8wa6YdI5BbEAKhaJWV7xlyxZcunSpQd+3sbHBtm3bcN999+H+++/H+fPn9VEm\nEdVhypQpuHjxYq3Fd4iai0FsYI8++ii6d++u/azRaPDRRx81+PtWVlZ47733EBISguHDhyMjI0Mf\nZRLRbaysrPDiiy8iPDxcdClkZhjEBmZtbV1rgY/333+/UQsGKBQKrFy5EitXrsT9998PtVqt6zKJ\nqA7Tp09Heno6/5sjnWIQCzBv3jy0adNG+/n8+fPYvn17o8eZM2cONm3ahICAAPzwww+6LJGI6tCq\nVSs8//zz7IpJpxjEArRv3x6zZ8+WHVu7dm2TJoE8/PDD+OabbxAaGop///vfuiqRiOoxZ84cxMXF\ncYc00hkucSlIdnY2+vbtKwvf/fv3Y/To0U0a7+TJk5g4cSIWLFiAsLAw2eIhRKRba9asQVpaGv7z\nn/+ILoXMAINYoODgYOzevVv7+eGHH8aePXuaPN7Zs2cxceJEjB8/HpGRkWjRgjc8iPShpKQETk5O\nSExMhKOjo+hyyMQxiAU6cOAARo0apf2sUCiQnp4OFxeXJo9ZXFyMoKAg9OzZE5s3b0arVq10USoR\n3ebll19GQUFBo956IKoLWyaBRowYgaFDh2o/S5KEqKioZo1pb2+Pn376CeXl5QgMDERZWVlzyySi\nOixatAg7duzAuXPnRJdCJo5BLFBdC3xER0ejqKioWeMqlUp89dVX6NOnD8aMGYMLFy40azwiqs3B\nwQEzZ87EO++8I7oUMnEMYsFCQkLQrVs3AECLFi0we/ZsVFVVNXtcKysrfPjhhwgMDMTw4cORnZ3d\n7DGJSO65555DdHQ0Ll68KLoUMmFWq1atWiW6CEtmZWUFOzs7jBw5Evn5+Vi8eDHc3d11MrZCocDo\n0aPRqlUrzJw5E2PGjEGXLl10MjYRAXZ2dsjJyUFaWhrGjh0ruhwyUZysZUQ+++wzfPXVV3rZ4WXX\nrl34v//7P3zxxRcYN26czscnslTZ2dnw9vZGTk4O7OzsRJdDJoi3po3ItGnTcOTIEZw4cULnYwcH\nB2Pnzp148skn8eWXX+p8fCJL5eTkhIkTJ2LDhg2iSyETxY7YyKxevRrnzp3Dpk2b9DJ+Wloa/P39\n8dxzz2Hx4sV6uQaRpUlLS8O4ceOQk5MDlUoluhwyMQxiI1NQUABXV1dkZGSgU6dOerlGXl4eJkyY\ngKCgIKxZs4arcBHpQHBwMMaOHYuFCxeKLoVMDIPYCM2dOxe9evXCyy+/rLdrXLp0CYGBgXBxccEn\nn3wCa2trvV2LyBIkJiZiypQpyMrK4kI61Ch8RmyElixZgg0bNuDatWt6u0bHjh3x66+/oqioCEFB\nQbhy5YrerkVkCYYNGwZXV1euP02NxiA2QgMGDIC7u7veJ1WpVCrs2rULXbt2xdixY1FYWKjX6xGZ\nu5UrV2LNmjWorq4WXQqZEAaxkVq6dCnWrVvXpK0RG6Nly5b45JNPMH78eIwYMQK5ubl6vR6RObv/\n/vvh4OCAnTt3ii6FTAiD2EiNHz8eVVVV+O233/R+LYVCgddffx0LFy7EyJEjkZycrPdrEpkjhUKB\nlStXIjw8XO+/RJP5YBAbKYVCgSVLlmDdunUGu+aCBQuwbt06PPjgg9i/f7/BrktkTgICAgBALwvz\nkHnirGkjptFo0Lt3b8TExKBfv34Gu+7+/fsxdepUfPDBBwgJCTHYdYnMxX//+1+sX78esbGxfD2Q\n7oodsRFTKpX45z//2eytERtrzJgx+Pnnn7FkyRJ88MEHBr02kTl49NFHcenSJcTExIguhUwAO2Ij\nd+HCBfTr1w+ZmZlwcHAw6LVzc3MxYcIEPPbYY/jXv/7F3+yJGmHz5s344osv8PPPP4suhYwcO2Ij\nd8899yA4OBgfffSRwa/dp08fxMbG4qeffsLcuXN1sj0jkaV48sknkZGRgcTERNGlkJFjR2wCUlNT\nMWHCBOTm5qJ169YGv/6VK1cQEhKCli1bYvv27VxLl6iB3n//ffzyyy/YvXu36FLIiLEjNgFubm4Y\nMGAAtm/fLuT6bdu2xZ49e9ChQweMGzcOly5dElIHkamZM2cOEhISkJaWJroUMmIMYhNhqAU+6mNt\nbY3o6GiMHDkSI0aMQF5enpA6iEyJUqnE4sWLERERAQAoLS0VXBEZI96aNhE1NTUYMGAANmzYgDFj\nxgitZf369XjnnXewd+9euLm5Ca2FyNiVlJSgR48e6NevHxQKBeLj4znxkWQYxCZk06ZN+O6777Bn\nzx7RpWDbtm1YtGgRvvrqK4wcOVL2s4KCAmyNjkZGSgqulJSgbbt2cHF3R+isWXrb2pHIGGk0Gtx/\n//1Qq9XaY7/88gseeOABgVWRsWEQmxCNRoNevXrh4MGDcHFxEV0OfvnlF0ybNg2bNm3CpEmToFar\nERURge/37cNkAF4VFbAFUAYgUanELknCQ/7+WLR8Oby8vARXT2QYQUFB+Pbbb7Wfx4wZY5Cla8l0\nMIhNzMsvv4yioiKjWWjjyJEjCAwMxNjRo/HrN99gmUaDUEmCfR3nFgOIVijwllKJ1ZGR+MfTTxu6\nXCKDi4+Ph6+vr+zYoUOHah0jy8UgNjF//fUX+vfvj+zsbHTo0EF0OQCAN157DR+sWoU/JAnOACoB\nzAfwC26ErxOAcAAT/z4/C8AElQrLGMZkIR544AFZFxwYGCjrksmyMYhN0MyZM+Hq6orly5eLLgVq\ntRpBo0fjQHk5nP8+Vg4gEsAsAD0AfA/gCQBpAHr+fU4WgJEqFb6NiYGnp6ehyyYyqF9//RXjxo2T\nHTt27BgGDRokqCIyJgxiE5ScnIyAgADk5uaiVatWQmuZPnkyPHfvxuK7/Gs0CMAqAMG3HFunUOBI\ncDD+zb1bycxJkgRfX18kJCRoj02dOhXbtm0TWBUZCwaxiXrggQcwa9YsTJ8+XVgNBQUFcO3VCzkV\nFXU+E77pAoA+AI4BuHWKWREAJxsbZOTlcTY1mb09e/bgkUce0X5WKBRIT083iomXJBYX9DBRohf4\nAICt0dEIBu4YwlUApgOYCXkIA0AHAMEKBbZGR+ulPiJjEhgYKHvvXpIkvPnmmwIrImPBIDZR/v7+\nuHr1Kv744w9hNWSkpGBYRUW9P5dwI4RbA3ivnnO8NBpkpKbqoToi49KiRYta8zq2bt3KVeqIQWyq\nWrRogcWLF2PdunXCarhSUgLbO/x8DoCLAL4GYFXPObYAyoqLdV0akVF67LHH4OzsrP1cVVWFd955\nR2BFZAwYxCZsxowZOHToELKysoRcv227diir52f/BJAOYA+AO00nKwNga3+nm9tE5sPKygrLli2T\nHfv4449RUFAgqCIyBgxiE6ZSqTBv3jxERUUJub6LuzsSbWxqHc8DsAk3Jmfdgxtdrx2AL+sYQ61U\nwoXrVZMFeeqpp9CtWzftZ41Gg/Xr1wusiETjrGkT9+eff2LgwIHIzs6GvYE7y4bOmq4PZ02TpYqK\nisLixYu1n+3s7HDmzBm0b99eYFUkCjtiE9e1a1cEBgZi06ZNBr92586d8ZC/P7Y0cSeZLQoFAgMC\nGMJkcebOnQsHBwft59LSUqNZtpYMjx2xGTh69CiCgoKQk5MDa2trg167rpW1GoIra5GlCw8Px8qV\nK7WfO3bsiDNnzqBNmzYCqyIR2BGbgcGDB8PZ2Rk7duww+LW9vLywOjISE1QqNHTK2M21pldHRjKE\nyWLNnz8fdnZ22s+XLl3Cxx9/LLAiEoVBbCZELvDxj6efxrLISIxUqbBOoUB9LyMVAVirUGAkN3wg\nQvv27bFgwQLZsbfffhvXrl0TVBGJwlvTZqKmpgb9+vXDp59+ipEjRwqpISkpCVEREfh69248XFOD\n+wHtfsTqv/cjDgwIwKLly9kJEwEoLCxEr169oNFotMc2bdqEefPmCayKDI1BbEY2bNiAX375BV9/\n/bXQOu677z6knzyJ1gBaAhjv7w+/MWMwY+ZMTswius3ixYtlryA6Ojri1KlTaNmypcCqyJAYxGbk\n6tWr6NWrFxISEuDk5CSsDnt7e1y+fFn7+fz587j33nuF1UNkzM6ePQtHR0dcv35de+zzzz/HtGnT\nBFZFhsRnxGakTZs2mDdvHt59911hNVy5ckUWwtbW1ujcubOweoiMXffu3REaGio7Fh4ejpqaGkEV\nkaGxIzYz586dg5ubG3JycoQsDpCeno7+/ftrP/fu3Ru5ubkGr4PIlGRlZcHV1VUWvrt375Ztm0jm\nix2xmenWrRsCAgLwySefCLl+fn6+7HOPHj2E1EFkSpydnTF16lTZsfDwcKHbnJLhMIjN0JIlS/Du\nu+/KnjkZCoOYqGlu3yIxMTERv/76q6BqyJAYxGZo6NCh6NOnD3bu3Gnwa98exN27dzd4DUSmyM3N\nDUFBQbJj4eHhgqohQ+L8eDO1dOlSvPHGG5g6dSoUTVwLuinOnj0r+8yOmKjhli9fjj179mg/79+/\nHwufeQaVeqNdAAARjElEQVTXSkpwpaQEbdu1g4u7O0JnzeKrgGaEHbGZCgwMRFFREQ4dOmTQ63p4\neGDkyJFQKpVwcHBgEBM1go+PD8aOHQsAUAGwAVD84YcY8vnneOi77zDk889xYtUquPTsiemTJ0Ot\nVgutl3TDatWqVatEF0G616JFC7Ro0QLbtm3DY489ZrDrDhs2DHZ2drh69SqOHDkCV1dXg3bkRKYu\n5ehRHE9IwCoA/wHwpCTBE4A7AE8Aj1RV4f+qqlBw6hQWfP457Dp0wFAvL5ElUzOxIzZjM2fOxO+/\n/27w14fOnj2r7YQZwkQNt2njRuz++GMcBvAcUO8+3/YAlkgSDpSX482wMGzauNFwRZLOMYjNWNu2\nbTFnzhyDL/CRn5/PSVpEjaRWq/FqWBh+vGVL0Q8AeOHGLerZdXzHGcCP5eV4NSwMSUlJhiqVdIxB\nbOYWLFiALVu2oKSkxGDXzM/P57NhokaKiojAMo1Gtq93NwAvA5hzh+85A3hBo0FURIRe6yP9YRCb\nuR49emDixIn49NNPDXbNW29NE9HdFRQU4Pt9+xB62wIekwAEAehwl++HShK+27sXhYWF+iqR9IhB\nbAGWLl2KqKgoVFVVGeR6vDVN1Dhbo6MRjPqfCd9NBwDBCgW2RkfrrigyGAaxBfD09ETPnj0Nsj1i\ndXU1/vrrL3Tr1k3v1yIyFxkpKRhWUdGsMbw0GmSkpuqoIjIkBrGFWLp0KdatW6f365w/fx4dO3ZE\nq1at9H4tInNRfPEibJs5hi2AsuJiXZRDBsYgthBBQUEoKChAXFycXq/D58NEd1ZVVYXk5GR89NFH\nmDVrFvr164cff/0VZc0ctwyArX1Tb26TSAxiC2FlZYVFixbpvSvm82EiuQsXLuCbb77B8uXLMWbM\nGNjb2+Pxxx9HfHw8fHx8sH37drz0r38h0cam1nerAVT8/ecqANf+/uu6qJVKuLi56e3vg/SH+xFb\nkLKyMvTu3RuHDx9G79699XKNtWvX4syZM4iKitLL+ETGrLKyEseOHUN8fLz2j+LiYvj4+Gj/GDZs\nGOxv61wLCgrg2qsXcioqZBO2Vv/9x63L4rwK4JXbrlsEwMnGBhl5eVyD2gQxiC3M888/j5qaGrzz\nzjt6GX/p0qXo2rUrwsLC9DI+kbGQJAlnz55FfHw84uLiEB8fj+TkZPTt21cWvC4uLmjR4u43H6dP\nngzP3buxuAn/l7xOocCR4GD8W8COa9R8DGILk5eXh8GDByM3Nxd2dnY6Hz8kJARTpkyptck5kakr\nLy/H4cOHZd1uVVWVNnB9fX3h6emJtm3bNml8tVqNoNGjceCWlbUaIgvASJUK38bEwNPTs0nXJrG4\nDaKF6dmzJx588EF89tlnWLx4sc7H5zNiMgeSJCE7O1sbuHFxcUhPT8fAgQPh4+ODKVOmIDIyEr17\n99bZeupeXl5YHRmJCbctc3knWQAmqFRYHRnJEDZh7IgtUGJiIqZOnYrMzEy0bKnb38W6deuGuLg4\n9OzZU6fjEulTaWkp1Gq19hZzfHw8VCqV7Bbz4MGDoVQq9V7Lpo0b8WpYGF7QaDBTkupc5KMIQLRC\ngbeVSqyOjMQ/nn5a73WR/jCILdTw4cOxZMkSTJkyRWdjXr9+HW3atMHVq1dhbW2ts3GJdKmmpgYn\nT56U3WLOzc3F4MGDtbeYvb29hS5Kk5SUhKiICHy3dy+CFQp4aTQ33hPGjdnRuyQJgQEBWLR8OTth\nM8AgtlA7d+7E2rVrERsbq7Mx8/Ly4Ovri3PnzulsTKLmunTpEhISErS3mBMTE9G5c2dZt+vu7m6U\nvzwWFhZia3Q0MlJTUVZcDFt7e7i4uWHGzJmcHW1GGMQWqrq6Gs7Ozti2bRu8vb11MmZsbCyee+45\nxMfH62Q8osa6fv06UlNTZd3uhQsX4OXlpQ1db29vhhgZFU7WslC3LvCxbds2nYzJVbXI0M6fPy8L\n3cOHD6NXr17w9fXFqFGj8MILL6B///6wsrISXSpRvdgRW7DS0lL06dMHR48e1cnkqsjISPz5559Y\nu3atDqojkrt27RqOHj0qm1B15coV2S1mLy8vtG/fXnSpRI3CjtiC2dnZYebMmXjvvffw9ttvN3u8\n/Px89OrVSweVkaWTJAlnzpyRdbupqalwdXWFj48PAgMD8frrr8PZ2Vlnrw8RicKO2MKdPn0aQ4cO\nxenTp2Fr27z9XyZPnownnngCISEhOqqOLMXVq1eRlJQkC14A8PX11Xa7Q4cORZs2bQRXSqR77Igt\nXO/evfHAAw9g8+bNePbZZ5s1Fp8RU0NIkoSMjAxZ6GZkZMDd3R0+Pj54/PHHsX79evTs2ZPdLlkE\ndsSEuLg4PPnkk8jMzGzWpJYuXbogKSlJ6PuXZHwuX76MxMREbegmJCTA1tZW+86uj48PPDw80Lp1\na9GlEgnBICYAN24BvvDCCwgODm7S9ysrK9G2bVtoNBrOULVg1dXVOHHihGxpyPz8fAwdOlT2+lCX\nLl1El0pkNBjEBADYsWMH3n33XRw4cKBJ3z99+jRGjRqFvLw8HVdGxqywsFB2i1mtVqNLly6yjRAG\nDhyo86VUicwJg5gAAFVVVXB2dsaOHTvg5eXV6O8fOHAAL774ok5X6iLjUllZiZSUFFnwXrx4Ed7e\n3rK9djt27Ci6VCKTwl9TCQDQsmVLPPvss1i3bh2++OKLRn+fuy6Zn3Pnzsn22j127BgcHR3h4+OD\nsWPHYsWKFejXr1+D9tolovqxIyatkpIS9OnTB8nJyY2e/fzWW2+hoKAAkZGReqqO9Emj0eDIkSOy\nbreioqLWXrv62MOayNKxIyatdu3aITQ0FO+//z7efPPNRn03Pz8fzs6N2c6cRJEkCbm5ubIJVSdO\nnMB9990HHx8fTJo0CWvWrIGjoyNfHyIyAHbEJJObmwsvLy+cPn0abdu2bfD3Jk2ahBkzZmDy5Ml6\nrI6aoqysDGq1WtbtWltbyxbLGDJkCFQqlehSiSwSO2KS6dOnD0aPHo3o6GgsWLCgwd/jM2LjUFNT\ng1OnTslCNysrCx4eHvD19cWMGTOwYcMG/m9FZETYEVMtsbGxCA0NxalTpxr8TvA999yDY8eO8f1Q\nAysqKtIulnFzr90OHTrINkIYNGgQWrVqJbpUIqoHg5hqkSQJPj4+WLFiBR555JG7nn/t2jXY2dlB\no9FwBq0eVVVVIS0tTdbt/vnnn/D09JQFb+fOnUWXSkSNwCCmOm3btg0bN25ETEzMXc/Nzs7GuHHj\nkJuba4DKLMeFCxdkoZuUlITu3bvLnu0OGDCAK5kRmTgGMdXp+vXrcHJywq5duzB06NA7nhsTE4OX\nXnqpyaty0Y3FMo4dOybba7ekpKTWYhn29vaiSyUiHeNkLaqTtbW1doGP//znP3c8Nz8/n7suNYIk\nScjPz5d1u8nJyejbty98fHwwceJErFq1Cn379uWtfiILwCCmes2dOxeOjo44d+7cHXdUYhDfWXl5\nOQ4fPix7b7e6ulp7izk8PByenp6Nel2MiMwHg5jq1b59ezz11FN4//33ERERUe95+fn56N+/vwEr\nM16SJCErK0vW7aanp2PgwIHw8fHBlClTEBkZid69e3OxDCICwGfEdBfZ2dnw9vbGmTNn0KZNmzrP\nCQoKwuzZszFp0iQDVydeaWmpbK/d+Ph4qFQq2V67gwcPho2NjehSichIsSOmO3JycsKoUaOwZcsW\nzJ8/v85zzOnWdE1NDU6ePAkbGxs4OTnV+bNbN0I4ffo0hgwZAh8fH8yePRsfffTRHW/jExHdjh0x\n3dWBAwcwZ84cpKen1zl5yMHBASdOnDDJ91cvXryIhIQEbTebmJiI0tJSLFy4EK+88kqtn3Xu3Fm2\nEYKbmxusra1F/20QkQljENNdSZKEYcOG4ZVXXsHDDz8s+5lGo4G9vT3Ky8uNfobv9evXkZqaKruN\nnJmZWee5rVu3RuvWrTFs2DBt8Hp7e8PBwcHAVRORueOtaborhUKBJUuWYO3atbWC+OzZs+jWrZtR\nhvD58+dlt5GTkpKg0Wga9N3q6mqcO3eOM5mJSO8YxNQgISEhWLZsGY4ePYrBgwdrjxvL8+GKigoc\nPXpU1u3m5eU1aaybazWXlJQwiIlI7xjE1CDW1tZYuHAh1q1bh61bt2qPi9h1SZIknDlzRha6R48e\nRWVlZaPHsrKygru7u2zZSGdnZ75aREQGw2fE1GDFxcVwdHRETEwMfvrhB2SkpOBoUhJgZYWpoaEI\nnTULnTp10vl1r169Wms/3QsXLjRprHvvvVcWukOHDq33tSwiIkNgEFODqdVqzH78cZw+cwaPWVvD\nq6ICtgDKACQqldglSXjI3x+Lli+Hl5dXk65RU1ODzMxMWeimpKSgpqam0WO1atVK+2rRzfDt0aMH\nu10iMioMYmqQTRs34tWwMLyg0WCmJKGurQeKAUQrFHhLqcTqyEj84+mn7zru5cuXZfvpJiQkoLi4\nuEk19u7dW7YdoIeHB1q3bt2ksYiIDIVBTHe1aeNGvBkWhh/Ly+H897FiALMB/AygE4BwAE/8/bMs\nABNUKiy7LYyrq6tx/PhxWbd78uTJJtWkUqng5eUlC9577723qX+LRETCMIjpjtRqNYJGj8aBW0IY\n+F/ofgbgCICHAMQBuLnidBaAEUolXnj9dVy6dEm7IMaVK1eaVIeLi4vs2e7AgQPRsiXnGhKR6WMQ\n0x1NnzwZnrt3Y/Et/5qUA7AHcALAzUUgQwF0w43O+KZIAK8AaNibu//Trl27WvvwduzYscl/D0RE\nxoxBTPUqKCiAa69eyKmokD0TPgZgBIBbe9u1AGIAfHPLsSLcCOeKO1xDoVBodya6OanK1dXVKBcI\nISLSB97bo3ptjY5GMFBrYtYVAHa3HbPDjdnTt+oA4BEA/wVw87c9BwcH2S1mLy8v2Nra6rp0IiKT\nwSCmemWkpGBYRe1+ti2A0tuOlQCoK07vB3DK1RVhL78MHx8fODo68vUhIqJbMIipXldKSuoMVxcA\nVQCy8b9nxMkABtRxri2A/n374sknn9RPkUREJo4P4qhebdu1q3W7GQBUACbjxkSscgAHAXwL4Kk6\nzi0DYGtf11vHREQEMIjpDlzc3ZFoY1Pnzz7AjRDuDGA6gA/xv1eXbqVWKuHi5qa3GomITB1nTVO9\n6ps13VBFAJxsbJCRl6eXNaiJiMwBO2KqV+fOnfGQvz+2NHFy1RaFAoEBAQxhIqI7YEdMd1Tfylp3\nkwVgpEqFb2Ni4Onpqa/yiIhMHjtiuiMvLy+sjozEBJUKWQ38zs21pldHRjKEiYjugkFMd/WPp5/G\nsshIjFSpsE6hQH17IxUBWKtQYGQdGz4QEVHdeGuaGiwpKQlRERH4bu9eBCsU8NJotPsRq//ejzgw\nIACLli9nJ0xE1EAMYmq0wsJCbI2ORkZqKsqKi2Frbw8XNzfMmDmTE7OIiBqJQUxERCQQnxETEREJ\nxCAmIiISiEFMREQkEIOYiIhIIAYxERGRQAxiIiIigRjEREREAjGIiYiIBGIQExERCcQgJiIiEohB\nTEREJBCDmIiISCAGMRERkUAMYiIiIoEYxERERAIxiImIiARiEBMREQnEICYiIhKIQUxERCQQg5iI\niEggBjEREZFADGIiIiKBGMREREQCMYiJiIgEYhATEREJxCAmIiISiEFMREQkEIOYiIhIIAYxERGR\nQAxiIiIigRjEREREAjGIiYiIBGIQExERCcQgJiIiEohBTEREJBCDmIiISCAGMRERkUAMYiIiIoEY\nxERERAIxiImIiARiEBMREQnEICYiIhKIQUxERCQQg5iIiEggBjEREZFADGIiIiKBGMREREQCMYiJ\niIgEYhATEREJxCAmIiISiEFMREQkEIOYiIhIIAYxERGRQAxiIiIigRjEREREAjGIiYiIBGIQExER\nCcQgJiIiEohBTEREJBCDmIiISCAGMRERkUAMYiIiIoEYxERERAIxiImIiARiEBMREQnEICYiIhLo\n/wGm2bFr6370RQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10ed31b00>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "graph = nx.DiGraph(data=data)\n",
    "nx.draw(graph, with_labels=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Suppose that we want to fit a [Bradley-Terry model](https://en.wikipedia.org/wiki/Bradley%E2%80%93Terry_model) on this data.\n",
    "`choix` provides several algorithms to do this; below, we use a maximum-likelihood inference algorithm called I-LSR."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 0.583  0.194 -0.    -0.194 -0.583]\n"
     ]
    }
   ],
   "source": [
    "params = choix.ilsr_pairwise(n_items, data)\n",
    "print(params)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The parameters can be thought of as the \"strength\" (or utility) of each item.\n",
    "It is possible to use them to rank the items: simply order the items by increasing parameter value."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ranking (worst to best): [4 3 2 1 0]\n"
     ]
    }
   ],
   "source": [
    "print(\"ranking (worst to best):\", np.argsort(params))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is also possible to use the parameters to predict outcomes of future comparisons."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Prob(1 wins over 4): 0.69\n"
     ]
    }
   ],
   "source": [
    "prob_1_wins, prob_4_wins = choix.probabilities([1, 4], params)\n",
    "print(\"Prob(1 wins over 4): {:.2f}\".format(prob_1_wins))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dealing with sparsity\n",
    "\n",
    "When the comparison graph is not connected, the maximum-likelihood estimate is not well defined.\n",
    "This happens for example when one item always wins, or always loses.\n",
    "In the following example, item $3$ always wins, and item $1$ always loses."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeIAAAFBCAYAAACrYazjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHYZJREFUeJzt3XtU1XW+//HXTuWWhHihq5dGf5gI6Ki7psxiZppKvI3m\nXJo5U1Tz6yBdNGPpWJnarDmkhzKtBo9n5kSeaf1mFmgry26/JlETGzZogmgi3SxFsQMhxkaD/T1/\nIIwbNgrK3p99eT7WmtXs/f2ye7Nai9d6v/fn+/nYLMuyBAAAjLjIdAEAAIQyghgAAIMIYgAADCKI\nAQAwiCAGAMAgghgAAIMIYgAADCKIAQAwiCAGAMAgghgAAIMIYgAADCKIAQAwiCAGAMAgghgAAIMI\nYgAADCKIAQAwiCAGAMAgghgAAIMIYgAADCKIAQAwiCAGAMAgghgAAIMIYgAADCKIAQAwiCAGAMAg\nghgAAIMIYgAADCKIAQAwiCAGAMAgghgAAIMIYgAADCKIAQAwiCAGAMAgghgAAIMIYgAADCKIAQAw\niCAGAMAgghgAAIMIYgAADCKIAQAwiCAGAMAgghgAAIMIYgAADCKIAQAwiCAGAMAgghgAAIMIYgAA\nDCKIAQAwiCAGAMAgghgAAIMIYgAADOptugAA51ZdXa11ubmqKC3Vibo69Y2JUXxysu6+5x4NGjTI\ndHkALoDNsizLdBEAPHM4HFqVlaVNb72lWZLsjY2KllQvqSgyUq9alqZMnqy5ixbJbrcbrhbA+SCI\nAT+1NidHSzIztdDp1N2WpVgP99RKyrXZtCIyUsuys3X/nDm+LhPABSKIAT+0NidHyzMz9U5Dg0Z0\n4f5KSbdFRWkhYQwEHIIY8DMOh0PTU1K07YwQflFSrqQySb+S9F8efq5S0qSoKL2+ZYsmTJjgm2IB\nXDBWTQN+ZlVWlhY6nW6d8JWSFku67yw/N0LSAqdTq7KyvFofgJ5FRwz4kerqao0cOlSfNjZ6/E54\nsaRD8twRS1KNpOEREao4eJDV1ECAoCMG/Mi63FzNlDyGcFf0lzTTZtO63NyeKwqAVxHEgB+pKC3V\ntY2NF/QZdqdTFWVlPVQRAG8jiAE/cqKuTtEX+BnRkupra3uiHAA+QBADfqRvTIzqL/Az6iVFx57v\ncBuArxHEgB8Zds012t67486zzZIaT/+zSdLJ0//fk62SisvK9Oc//1mffPKJWI8J+DdWTQN+YuPG\njcrIyND/HDqkw3JfsLXs9P9sZ7y3RNKT7T6jRtL3wsO1JCtLxcXFKigo0EUXXaSUlBT98Ic/VEpK\niq6++mrZbDYB8A8EMWDYF198oYcfflgbN26UJEVJekrSo+fxWSttNu2cOVP/vX69JMmyLFVWVqqg\noECbN29WQUGBevfurZSUlLZwHjZsGMEMGEQQA4Z89913WrlypZYtW6aGhga3a5dIKpG6tL1lq67s\nrGVZlg4cOKCCgoK2cA4LC3PrmIcNG3aevxGA80EQAwZs27ZNc+bMUXl5eYdrvXr10o9TUlRZWKh3\n2u2w1Znz3WvasixVVFS4BXNERIRbMA8dOrTrvxiAbiOIAR/6+uuvtWDBAr300kser19//fVas2aN\nkpOT205fWuB0Kq2T05dq1HL60r/30OlLlmVp//79bcFcUFCgyMhIt2AeMmTIBf07ALgjiAEfcLlc\neumll7RgwQLV1NR0uB4bG6sVK1bo3nvv1UUX/fNhhuLiYq3KytIbb76pmTab7E5n23nEjtPnEU9N\nTdXcRYu8ctCDZVn6+OOP3YL54osvdgvmwYMH9/i/FwglBDHgZWVlZUpPT1dhYaHH62lpaVqxYsVZ\n94Y+duyY1uXmqqKsTPW1tYqOjVV8UpLuSkvz6Z7SlmVp3759bsEcHR3tFsxXXXWVz+oBggFBDHjJ\niRMntGzZMq1cuVLNzR2f+k1ISFBOTo5uuukmA9X1DMuytHfvXrdgjomJcQvmK6+80nSZgF8jiIEe\nZlmWXnvtNT388MP68ssvO1yPjIzUkiVL9MgjjygsLMxAhd7jcrk6BHNsbGxbMN98880EM9AOQQz0\noM8//1wPPfSQ3njjDY/Xp02bptWrV4fMI0Iul0vl5eVtobxlyxb179/fLZivuOIK02UCRhHEQA84\ndeqUnn32WT311FNyOp0drg8ePFjPP/+8ZsyYYaA6/+FyubRnzx63YB44cKBbMF9++eWmywR8iiAG\nLlBlZaVmzJihvXv3drjWu3dvPfLII3ryySfVt29fA9X5N5fLpbKyMrdgjouLcwvmyy67zHSZgFcR\nxMAFamho0MiRI/XVV1+5vX/jjTcqJydHiYmJhioLPC6XS6WlpW3BvHXrVl166aVtW3KmpKTo0ksv\nNV0m0KMIYuACnDx5UtnZ2Vq+fLnq61sOMBwwYIBWrFihtLQ0t2eC0X3Nzc0dgvnyyy93C+a4uDjT\nZQIXhCAGztPmzZs1Z84cxcfHa/Xq1crMzFS/fv309NNPa+DAgabLC0rNzc3avXt3WzBv27ZNV1xx\nRVso33zzzQQzAg5BDHTT0aNHlZmZqa1bt2r16tVtC7CamprU28NZwvCe5uZmffTRR27BfNVVV7kF\nsy83PAHOB0EMdFFzc7PWrl2rJUuWKC0tjQVYfqi5uVm7du1qC+YPPvhAgwcPdgtmphXwNwQx0AW7\ndu1Senq6wsLCWIAVQJqamjoE89ChQ92CecCAAabLRIgjiIGzOH78uBYvXqy//vWvysrKYgFWgGtq\natLOnTvbgnn79u0aNmyYWzD379/fdJkIMQQx4IFlWcrLy9P8+fN1++23swArSDU1NamkpMQtmL/3\nve+1BfNNN91EMMPrCGKgncrKSj344IM6dOiQ1qxZo4kTJ5ouCT7y3XffuQVzYWGhhg8f7hbMsbGe\nToYGzh9BDJx28uRJLV++XKtXr9bChQs1b9489enTx3RZMOi7775TcXFxWzDv2LFDI0aMcAvmfv36\nmS4TAY4gBiT9/e9/V0ZGhhISErRq1SoNGTLEdEnwQ6dOneoQzPHx8W3BPGnSpA7BfODAAUVHR7NV\nJzpFECOkHTlyRPPnz1dhYaGef/55TZs2zXRJCCCnTp2Sw+FoC+YPP/xQI0eOdAvme++9Vxs2bNA1\n11zDVp3wiCBGSGpubtaaNWu0dOlS/fa3v9UTTzyhiy++2HRZCHCnTp1SUVGRWzCfPHlSTU1NHe4d\nNWoUW3VCEkGMEFRSUqL09HRFRUXpj3/8o0aPHm26JASpkpISTZgwoUv3JiQkeG2rzurqaq3LzVVF\naalO1NWpb0yM4pOTdfc997DzmB8giBEy6urq9MQTTygvL0/Lly/XXXfdJZvNZrosBLHCwkLNnz9f\nxcXFam5u7tbPjh49+oK36nQ4HFqVlaVNb72lWZLsjY2KllQvqSgyUq9alqZMnqy5ixbJbrd3+/PR\nMwhiBD3LsvS3v/1Njz76qKZMmaKsrCx2U4JP1dfXa/v27dq8ebMKCgpUUlLS7WBOTEzs1lada3Ny\ntCQzUwudTt1tWfL00FWtpFybTSsiI7UsO1v3z5nTrZrQMwhiBLUDBw4oIyND1dXVysnJ0Q033GC6\nJEDHjx/vEMwul6tbn5GUlNTpVp1rc3K0PDNT7zQ0aEQXPqtS0m1RUVpIGBtBECMoNTY26umnn9YL\nL7ygxx57TA8//DAnI8FvHT9+XB988EFbMO/cubPbwZycnKyUlBRdeeWVWrl0qbY5nW4hXCvpXkn/\nX9IgSf8m6c4zrldKmhQVpde3bOny99roGQQxgs67776rBx54QMnJyXruuec0ePBg0yUB3VJXV+cW\nzLt27epyMEdJekrSo+3ebw3d/5K0U9IUSTskjTrjnpU2m3bOnKn/Xr/+wn4BdAtBjKBx+PBhzZ8/\nX0VFRXrhhReUmppquiSgR3zzzTcdgrmzP90Rkg5Lbt8JN5x+vVfS8NPv3S3pSrV0xq1qJA2PiFDF\nwYOspvYhjpFBwGtubtbq1as1ZswYDR8+XHv27CGEEVT69eunqVOn6plnnlFJSYlqamq0ceNGPfLI\nI/r+97/ftvrfJumnUoeFWRWS+uifISxJYySVt7uvv6SZNpvW5eZ65feAZ3xphoDmcDiUnp6uSy65\nRFu3btWoUaPO/UNAgOvXr5+mTZvWthNcbW2ttm3bpqULF+rmjz/ucP8JSZe0e+8StTzG1J7d6dRH\nZWU9XTLOgo4YAembb77RAw88oOnTp2vevHl6//33CWGErNjYWE2fPl3XjBihaA/X+0o63u69Osnj\nvdGS6mtre7pEnAVBjIBiWZZeeeUVJSQkqLm5WXv37tVvfvMbNuYAJPWNifHY5cZLapL0yRnv7Zbk\naU+5eknRHPXoU4ymETD279+vjIwM1dTU6NVXX9V1111nuiTAr8QnJ6to/XqlNza6vR8laZakJyX9\np1pWTb8uqdDDZzgiIzU6KcnbpeIMdMTwe06nU4sXL9bEiRM1bdo0ORwOQhjw4K60NL2qlmeG23tR\nLaun4yT9i6Q1cn90SWpZNf2qZemutDRvlol2CGL4tbfffluJiYnav3+/du/erXnz5rExB9CJuLg4\nTZk8WS97+KomVtKralm49bmkX3j4+ZdtNk1NTeXRJR/jOWL4pUOHDmnevHnauXOnXnzxRd1+++2m\nSwICgsPh0PSUFG3r4vaWrdhZyxw6YviVpqYmPffccxozZoxGjRqlPXv2EMJAN9jtdi3LztZtUVGq\n7OLPtO41vSw7mxA2gBkf/MY//vEPpaenq3///tq+fbtGjhxpuiQgILUe3DApM1MLnE6ldXL6Uo1a\nTl/6d05fMorRNIyrra3VokWLtHHjRmVnZ+vOO+/kcSSgBxQXF2tVVpbeePNNzbTZZHc6284jdpw+\nj3hqaqrmLlpEJ2wQQQxjLMvSX/7yFy1YsECzZs3SH/7wB/Xr1890WUDQOXbsmNbl5qqirEz1tbWK\njo1VfFKS7kpLY2GWHyCIYcS+ffuUkZGh48ePa82aNbLb7aZLAgAjWKwFn2poaNDjjz+um266SbNm\nzVJRUREhDCCksVgLPrNp0yY99NBDuvbaa7V7925dccUVpksCAOMIYnjdV199pblz56q0tFRr1qzR\nrbfearokAPAbjKbhNU1NTXr22Wc1duxYJSUlqaysjBAGgHboiOEVO3bsUHp6uuLi4lRYWKj4+HjT\nJQGAXyKI0aNqamr0u9/9Tps2bdIzzzyjX/ziFzwTDABnwWgaPcKyLL388stKSEhQeHi49u7dq1/+\n8peEMACcAx0xJEnV1dUtD/yXlupEXZ36xsQoPjlZd99zzzkf+C8vL1dGRoYaGhq0adMmjR8/3kdV\nA0DgY0OPEOdwOLQqK0ub3npLsyTZGxvbtsArOr0F3pTJkzV30aIOz/s2NDTo97//vf70pz9p6dKl\nSk9PV69evUz8GgAQsAjiELY2J0dLMjO10OnU3Z1sCl+rlk3hV7TbFP7111/XQw89pIkTJ+qZZ57R\nZZdd5tPaASBYEMQham1OjpZnZuqdLp5Z2npM2v997DH9o7hYe/fu1YsvvqhbbrnF26UCQFAjiEOQ\np4PDT0nKkPSeWrrg4ZL+TdKZJwFXSpog6Zf/+q9atWqVwsPDfVk2AAQlVk2HoFVZWVrodLp1wk2S\nhkjaJqlO0u8l/VzSwTPuGSHpSZtN3x47RggDQA+hIw4x1dXVGjl0qD5tbPT4nfCZxkhaKmnmGe/V\nSBoeEaGKgwc5Pg0AegAdcYhZl5urmdI5Q/iopAOSRrd7v7+kmTab1uXmeqE6AAg9BHGIqSgt1bWN\njWe9p0nSv0hKk+RpY0q706mKsrKeLw4AQhBBHGJO1NUp+izXLbWEcLik5zu5J1pSfW1tT5cGACGJ\nnbVCTN+YGNWf5fp9kr6W9KakzrbmqJcUHXuu4TYAoCvoiENMfHKyiiIiPF5Ll/SxpI2Sws7yGY7I\nSMUnJXmhOgAIPayaDjGdrZo+KGmYpAj9sxO2SfoPSXeecR+rpgGgZ9ERh5i4uDhNmTxZL7c7FWmI\nJJekBrWMnuslHZd7CEvSyzabpqamEsIA0EPoiEOQp521uqJS0qSoKL2+ZYsmTJjgrfIAIKTQEYcg\nu92uZdnZui0qSpVd/JnWvaaXZWcTwgDQgwjiEHX/nDlakJ2tCTabstWyv7QnNZKetdk0KSpKC884\nfQkA0DMYTYew4uJi2e12Rarl+eEZkm6W2s4jdpw+j3hqaqrmLlpEJwwAXsBzxCEsLy9PkuQ8/Xrv\n6NGKGTdO9bW1io6N1eikJC1PS2NhFgB4ER1xiLIsS8OHD9dnn33W9t4rr7yiX/3qVwarAoDQQxCH\nqJKSErdRc3h4uI4dO6bo6LNtgAkA6Gks1gpRrWPpVrfffjshDAAGEMQhyLKsDkH8s5/9zFA1ABDa\nGE2HoJ07d2r8+PFtr8PDw1VdXa1LLrnEYFUAEJroiENQ+274tttuI4QBwBCCOMQwlgYA/8JoOsTs\n2rVL48aNa3sdFham6upqxcTEGKwKAEIXHXGI8TSWJoQBwByCOIQwlgYA/8NoOoTs3r1bY8eObXvd\np08fHTt2jI4YAAyiIw4h7bvhW2+9lRAGAMMI4hDBWBoA/BOj6RBRWlqqMWPGtL3u06ePqqur1a9f\nP4NVAQDoiENE+274Jz/5CSEMAH6AIA4BjKUBwH8xmg4BZWVlSk5Obnvdp08fHT16VLGxsQarAgBI\ndMQhoX03fMsttxDCAOAnCOIgx1gaAPwbo+kgt2fPHiUlJbW97t27t44ePar+/fsbrAoA0IqOOMh5\nGksTwgDgPwjiIMdYGgD8G6PpIFZeXq7ExMS217169dLRo0c1YMAAg1UBAM5ERxzE8vPz3V7/+Mc/\nJoQBwM8QxEGMsTQA+D9G00Fq3759SkhIaHvdq1cvHTlyRAMHDjRYFQCgPTriINW+G/7Rj35ECAOA\nHyKIg9Svf/1r3XfffYqIiJDEWBoA/BWj6SD24IMP6vLLL9fPf/5zxcXFKSYmxnRJAIB2COIg5XK5\ndNVVV6mgoEDx8fGmywEAdILRdJDavn27Bg0aRAgDgJ8jiINUXl4e3wsDQABgNB2EXC6XBg8erPff\nf18jR440XQ4A4CzoiINQYWGhBgwYQAgDQAAgiINQXl6eZs+ebboMAEAXMJoOMq1j6ffee0+jRo0y\nXQ4A4BzoiIPMjh07FBsbSwgDQIAgiIMMq6UBILAwmg4iLpdLQ4YM0bvvvut24AMAwH/REQeRDz/8\nUDExMYQwAAQQgjiIMJYGgMDDaDpIuFwuDR06VG+//bZGjx5tuhwAQBfREQeJoqIiRUdHE8IAEGAI\n4iDBWBoAAhOj6SBgWZaGDRumTZs2KTEx0XQ5AIBuoCMOAkVFRYqKimIsDQABiCAOAq17S9tsNtOl\nAAC6idF0gLMsS1dffbU2btyo5ORk0+UAALqJjjjAORwOhYeHKykpyXQpAIDzQBAHuNbV0oylASAw\nMZoOYK1j6ddee01jxowxXQ4A4DzQEQew4uJihYWF8d0wAAQwgjiA5efnM5YGgADHaDpAWZal4cOH\na8OGDRo7dqzpcgAA54mOOEDt3LlTvXr14rthAAhwBHGAYrU0AASH3qYLQPdZlqW8vDzl5+ebLgUA\ncIHoiAPQrl27JInvhgEgCBDEAYixNAAED4I4wLSOpTl7GACCA0EcYD766CO5XC6NGzfOdCkAgB5A\nEAcYxtIAEFwI4gDCWBoAgg9BHEB2796tpqYmjR8/3nQpAIAeQhAHEPaWBoDgQxAHCMbSABCcCOIA\nUVZWplOnTmnChAmmSwEA9CCCOEDk5eVp9uzZjKUBIMgQxAGgdSw9e/Zs06UAAHoYQRwA9uzZI6fT\nqWuvvdZ0KQCAHkYQBwDG0gAQvAhiP8dqaQAIbgSxnysvL9e3336r6667znQpAAAvIIj9HGNpAAhu\nBLGfYywNAMGNIPZj5eXlqq+vZywNAEGMIPZj+fn5mj17ti66iP9MABCs+AvvxxhLA0DwI4j91L59\n+/TNN9/oBz/4gelSAABeRBD7qdbV0oylASC48VfeT7G3NACEBoLYD3388ceqqanRDTfcYLoUAICX\nEcR+KC8vT3fccQdjaQAIAfyl90OslgaA0EEQ+5n9+/fr66+/1sSJE02XAgDwAYLYzzCWBoDQwl97\nP8NYGgBCC0HsRyoqKlRdXc1YGgBCCEHsR/Lz83XHHXeoV69epksBAPgIQexHGEsDQOghiP1EZWWl\nqqqqdOONN5ouBQDgQwSxn8jLy9OsWbMYSwNAiCGI/QRjaQAITTbLsizTRYS6Tz75RDfccIMOHz5M\nRwwAIYaO2A8wlgaA0EUQ+wHG0gAQuhhNG/bpp5/q+uuv16FDh9S7d2/T5QAAfIyO2LC8vDzNnDmT\nEAaAEEUQG8ZYGgBCG6Npgz777DNdd911Onz4MB0xAIQoOmKD8vPzGUsDQIgjiA1iLA0AYDRtyOef\nfy673a6qqio6YgAIYXTEhuTn5+unP/0pIQwAIY4gNoSxNABAYjRtxBdffKHx48erqqpKffr0MV0O\nAMAgOmIDWsfShDAAgCA2gLE0AKAVo2kfO3jwoMaNG8dYGgAgiY7Y5/Lz8zVjxgxCGAAgiSD2ufz8\nfMbSAIA2jKZ96Msvv9TYsWN15MgROmIAgCQ6Yp9av349Y2kAgBuC2IdYLQ0AaI/RtI989dVXSk5O\n1pEjRxQWFma6HACAn6Aj9pH169dr+vTphDAAwA1B7COMpQEAnjCa9oFDhw4pKSmJsTQAoAM6Yh9Y\nv369pk2bRggDADogiH2AsTQAoDOMpr3s8OHDSkxMVFVVlcLDw02XAwDwM3TEXrZ+/XpNnTqVEAYA\neEQQexl7SwMAzobRtBdVVVUpISFBR44coSMGAHhER+xFGzZsYCwNADgrgtiL8vLyNHv2bNNlAAD8\nGKNpLzly5IhGjRqlqqoqRUREmC4HAOCn6Ii9ZMOGDUpNTSWEAQBnRRB7CZt4AAC6gtG0Fxw9elQj\nR45UVVWVIiMjTZcDAPBjdMRe0DqWJoQBAOfS23QBga66ulrrcnNVUVqqE3V16hsTo807dujxxYtN\nlwYACACMps+Tw+HQqqwsbXrrLc2SZG9sVLSkeklbJL0dHq4pqamau2iR7Ha72WIBAH6LID4Pa3Ny\ntCQzUwudTt1tWYr1cE+tpFybTSsiI7UsO1v3z5nj6zIBAAGAIO6mtTk5Wp6ZqXcaGjSiC/dXSrot\nKkoLCWMAgAcEcTc4HA5NT0nRtnYh/BtJ70lqkDRQ0r2SHj/jeqWkSVFRen3LFk2YMMF3BQMA/B6r\nprthVVaWFjqdHTrhRZI+k1Qn6S1Jz0t654zrIyQtcDq1KivLN4UCAAIGHXEXVVdXa+TQofq0sdHj\nd8Kt9ku6RdJrksad8X6NpOEREao4eFCDBg3yZqkAgABCR9xF63JzNVPqNIQfkHSxpES1jKXHtbve\nX9JMm03rcnO9ViMAIPAQxF1UUVqqaxsbO73+oqQTavmu+AlJDg/32J1OVZSVeadAAEBAIoi76ERd\nnaLPcY9N0s2Sfibp/3m4Hi2pvra2p0sDAAQwgriL+sbEqL6L9zZJivLwfr2k6NizfcMMAAg1BHEX\nxScnq8jDkYbHJP1N0reSXGpZLZ0naYaHz3BERio+KcmbZQIAAgyrpruos1XTX0uaLalUkiXp/0ha\nLGlau59n1TQAwBM64i6Ki4vTlMmT9bLN5vb+QEkFagnaWklF6hjCkvSyzaapqamEMADADR1xN3S2\ns9a5sLMWAKAzdMTdYLfbtSw7W7dFRamyiz/Tutf0suxsQhgA0AFB3E33z5mjhdnZmhQVpZU2mzp7\nGKlG0rM2myZx4AMA4CwYTZ+n4uJircrK0htvvqmZNpvsTmfbecSOyEi9almaevo8YjphAEBnCOIL\ndOzYMa3LzVVFWZnqa2sVHRur+KQk3ZWWxsIsAMA5EcQAABjEd8QAABhEEAMAYBBBDACAQQQxAAAG\nEcQAABhEEAMAYBBBDACAQQQxAAAGEcQAABhEEAMAYBBBDACAQQQxAAAGEcQAABhEEAMAYBBBDACA\nQQQxAAAGEcQAABhEEAMAYBBBDACAQQQxAAAGEcQAABhEEAMAYBBBDACAQQQxAAAGEcQAABhEEAMA\nYBBBDACAQQQxAAAGEcQAABhEEAMAYBBBDACAQQQxAAAGEcQAABhEEAMAYBBBDACAQf8LEVaKYaHr\nCmgAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11260a748>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "n_items = 4\n",
    "data = [(3, 2), (2, 1), (1, 0)]\n",
    "\n",
    "graph = nx.DiGraph(data=data)\n",
    "nx.draw(graph, with_labels=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In these cases, most of the estimators will fail by default."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/maystre/Documents/choix/choix/utils.py:18: RuntimeWarning: divide by zero encountered in log\n",
      "  params = np.log(weights)\n",
      "/Users/maystre/Documents/choix/choix/utils.py:19: RuntimeWarning: invalid value encountered in subtract\n",
      "  return params - params.mean()\n"
     ]
    },
    {
     "ename": "RuntimeError",
     "evalue": "Did not converge after 100 iterations",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mRuntimeError\u001b[0m                              Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-8-ad5bf7166344>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mchoix\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0milsr_pairwise\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mn_items\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdata\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m/Users/maystre/Documents/choix/choix/lsr.py\u001b[0m in \u001b[0;36milsr_pairwise\u001b[0;34m(n_items, data, alpha, initial_params, max_iter, tol)\u001b[0m\n\u001b[1;32m    105\u001b[0m     \"\"\"\n\u001b[1;32m    106\u001b[0m     return _ilsr(\n\u001b[0;32m--> 107\u001b[0;31m             n_items, data, alpha, initial_params, max_iter, tol, lsr_pairwise)\n\u001b[0m\u001b[1;32m    108\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    109\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/Users/maystre/Documents/choix/choix/lsr.py\u001b[0m in \u001b[0;36m_ilsr\u001b[0;34m(n_items, data, alpha, params, max_iter, tol, lsr_fun)\u001b[0m\n\u001b[1;32m     30\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mconverged\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mparams\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     31\u001b[0m             \u001b[0;32mreturn\u001b[0m \u001b[0mparams\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 32\u001b[0;31m     \u001b[0;32mraise\u001b[0m \u001b[0mRuntimeError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"Did not converge after {} iterations\"\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mformat\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmax_iter\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     33\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     34\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mRuntimeError\u001b[0m: Did not converge after 100 iterations"
     ]
    }
   ],
   "source": [
    "choix.ilsr_pairwise(n_items, data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The problem can be solved by adding a little bit of regularization as follows."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-4.159, -0.979,  1.483,  3.655])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "choix.ilsr_pairwise(n_items, data, alpha=0.01)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}