ch_02/1-pandas_data_structures.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Pandas Data Structures Overview\n",
    "In this section, we will discuss the `Series`, `Index`, and `DataFrame` classes. To do so, we will read in a snippet of the CSV file we will work with later. Don't worry about that part yet, though.\n",
    "\n",
    "## About the Data\n",
    "In this notebook, we will be working with 5 rows from the earthquake data collected over September 18, 2018 - October 13, 2018 (obtained from the US Geological Survey (USGS) using the [USGS API](https://earthquake.usgs.gov/fdsnws/event/1/))\n",
    "\n",
    "## Working with NumPy Arrays\n",
    "Let's read in a short CSV file (using `numpy`) for some sample data. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([('2018-10-13 11:10:23.560', '262km NW of Ozernovskiy, Russia', 'mww', 6.7, 'green', 1),\n",
       "       ('2018-10-13 04:34:15.580', '25km E of Bitung, Indonesia', 'mww', 5.2, 'green', 0),\n",
       "       ('2018-10-13 00:13:46.220', '42km WNW of Sola, Vanuatu', 'mww', 5.7, 'green', 0),\n",
       "       ('2018-10-12 21:09:49.240', '13km E of Nueva Concepcion, Guatemala', 'mww', 5.7, 'green', 0),\n",
       "       ('2018-10-12 02:52:03.620', '128km SE of Kimbe, Papua New Guinea', 'mww', 5.6, 'green', 1)],\n",
       "      dtype=[('time', '<U23'), ('place', '<U37'), ('magType', '<U3'), ('mag', '<f8'), ('alert', '<U5'), ('tsunami', '<i8')])"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "data = np.genfromtxt(\n",
    "    'data/example_data.csv', delimiter=';', \n",
    "    names=True, dtype=None, encoding='UTF'\n",
    ")\n",
    "data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can find the dimensions with the `shape` attribute:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(5,)"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can find the data types with the `dtype` attribute:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dtype([('time', '<U23'), ('place', '<U37'), ('magType', '<U3'), ('mag', '<f8'), ('alert', '<U5'), ('tsunami', '<i8')])"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.dtype"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Each of the entries in the array is a row from the CSV file. NumPy arrays contain a single data type (unlike lists, which allow mixed types); this allows for fast, vectorized operations. When we read in the data, we got an array of `numpy.void` objects, which are created to store flexible types. This is because NumPy has to store several different data types per row: four strings, a float, and an integer. This means we can't take advantage of the performance improvements NumPy provides for single data type objects.\n",
    "\n",
    "Say we want to find the maximum magnitude&mdash;we can use a **[list comprehension](https://www.python.org/dev/peps/pep-0202/)** to select the third index of each row, which is represented as a `numpy.void` object. This makes a list, meaning that we can take the maximum using the `max()` function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "9.74 µs ± 177 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)\n"
     ]
    }
   ],
   "source": [
    "%%timeit\n",
    "max([row[3] for row in data])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we, instead, create a NumPy array for each column, this operation is much easier (and more efficient) to perform. We can use a **[dictionary comprehension](https://www.python.org/dev/peps/pep-0274/)** to make a dictionary where the keys are the column names and the values are NumPy arrays of the data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'time': array(['2018-10-13 11:10:23.560', '2018-10-13 04:34:15.580',\n",
       "        '2018-10-13 00:13:46.220', '2018-10-12 21:09:49.240',\n",
       "        '2018-10-12 02:52:03.620'], dtype='<U23'),\n",
       " 'place': array(['262km NW of Ozernovskiy, Russia', '25km E of Bitung, Indonesia',\n",
       "        '42km WNW of Sola, Vanuatu',\n",
       "        '13km E of Nueva Concepcion, Guatemala',\n",
       "        '128km SE of Kimbe, Papua New Guinea'], dtype='<U37'),\n",
       " 'magType': array(['mww', 'mww', 'mww', 'mww', 'mww'], dtype='<U3'),\n",
       " 'mag': array([6.7, 5.2, 5.7, 5.7, 5.6]),\n",
       " 'alert': array(['green', 'green', 'green', 'green', 'green'], dtype='<U5'),\n",
       " 'tsunami': array([1, 0, 0, 0, 1])}"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "array_dict = {\n",
    "    col: np.array([row[i] for row in data])\n",
    "    for i, col in enumerate(data.dtype.names)\n",
    "}\n",
    "array_dict"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Grabbing the maximum magnitude is now simply a matter of selecting the `mag` key and calling the `max()` method. This is nearly twice as fast as the list comprehension implementation when dealing with just 5 entries, imagine how much worse the first attempt will perform on large data sets:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5.22 µs ± 100 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)\n"
     ]
    }
   ],
   "source": [
    "%%timeit\n",
    "array_dict['mag'].max()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "However, this representation has other issues. Say we wanted to grab all the information for the earthquake with the maximum magnitude, how would we go about that? We would need to find the index of the maximum and then for each of the keys in the dictionary grab that index:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['2018-10-13 11:10:23.560', '262km NW of Ozernovskiy, Russia',\n",
       "       'mww', '6.7', 'green', '1'], dtype='<U31')"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.array([\n",
    "    value[array_dict['mag'].argmax()] \n",
    "    for key, value in array_dict.items()\n",
    "])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The result is now a NumPy array of strings (our numeric values were converted), and we are now in the format from earlier. Also, consider trying to sort the data by magnitude from smallest to largest. In the first representation, we would have to sort the rows by examining the 3rd index. With the second representation, we would have to determine the order for the indices from the `mag` column, and then sort all the other arrays with those same indices. Clearly, working with several NumPy arrays of different data types at once is a bit cumbersome. However, `pandas` builds on top of NumPy arrays to make this easier. Let's start our exploration of `pandas` with an overview of the data structures.\n",
    "\n",
    "## `Series`\n",
    "The `Series` class provides a data structure for arrays of a single type with some additional functionality."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0          262km NW of Ozernovskiy, Russia\n",
       "1              25km E of Bitung, Indonesia\n",
       "2                42km WNW of Sola, Vanuatu\n",
       "3    13km E of Nueva Concepcion, Guatemala\n",
       "4      128km SE of Kimbe, Papua New Guinea\n",
       "Name: place, dtype: object"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "place = pd.Series(array_dict['place'], name='place')\n",
    "place"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here are some commonly used attributes with `Series` objects:\n",
    "\n",
    "|Attribute | Returns |\n",
    "| --- | --- |\n",
    "| `name` | The name of the `Series` object |\n",
    "| `dtype` | The data type of the `Series` object |\n",
    "| `shape` | Dimensions of the `Series` object in a tuple of the form `(number of rows,)` |\n",
    "| `index` | The `Index` object that is part of the `Series` object |\n",
    "| `values` | The data in the `Series` object |\n",
    "\n",
    "For the most part, `pandas` objects use NumPy arrays for their internal data representations. However, for some data types, `pandas` builds upon NumPy to create its own [arrays](https://pandas.pydata.org/pandas-docs/stable/reference/arrays.html). For this reason, depending on the data type, `values` can either be a `pandas.array` or `numpy.array` object. Therefore, if we need to ensure we get a specific type back, then it is recommended to use the `array` attribute or `to_numpy()` method, respectively, instead of `values`.\n",
    "\n",
    "Now let's see some examples using these attributes.\n",
    "\n",
    "### Getting the name of the series\n",
    "The NumPy array held the name of the data in the `dtype` attribute; here, we can access it directly: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'place'"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "place.name"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Getting the data type\n",
    "A `Series` object holds a single data type. Here it is `'O'` for object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dtype('O')"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "place.dtype"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Getting the dimensions of the series\n",
    "Just as with NumPy, we can use `shape` to get the dimensions as `(rows, columns)`. `Series` objects are a single column, so they only have values for the rows dimension. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(5,)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "place.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Isolating the values from the series\n",
    "This `Series` object is storing its values as a NumPy array:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['262km NW of Ozernovskiy, Russia', '25km E of Bitung, Indonesia',\n",
       "       '42km WNW of Sola, Vanuatu',\n",
       "       '13km E of Nueva Concepcion, Guatemala',\n",
       "       '128km SE of Kimbe, Papua New Guinea'], dtype=object)"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "place.values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## `Index`\n",
    "The addition of the `Index` class makes the `Series` class more powerful than a NumPy array. We can get the index from the `index` attribute of a `Series` object:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RangeIndex(start=0, stop=5, step=1)"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "place_index = place.index\n",
    "place_index"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As with `Series` objects, we can access the underlying data via the `values` attribute. Note that this `Index` object is also built on top of a NumPy array:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 1, 2, 3, 4])"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "place_index.values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here are some commonly used attributes with `Index` objects:\n",
    "\n",
    "|Attribute | Returns |\n",
    "| --- | --- |\n",
    "| `name` | The name of the `Index` object |\n",
    "| `dtype` | The data type of the `Index` object |\n",
    "| `shape` | Dimensions of the `Index` object |\n",
    "| `values` | The data in the `Index` object |\n",
    "| `is_unique` | Check if the `Index` object has all unique values |\n",
    "\n",
    "We can check the type of the underlying data, just like with a `Series` object:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dtype('int64')"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "place_index.dtype"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Same for the dimensions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(5,)"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "place_index.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can check if the values are unique:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "place_index.is_unique"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With NumPy we can perform arithmetic operations element-wise between arrays:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 1, 2])"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.array([1, 1, 1]) + np.array([-1, 0, 1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Pandas supports this as well, and the index determines how element-wise operations are performed. With addition, only the matching indices are summed:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     NaN\n",
       "1     2.5\n",
       "2     7.5\n",
       "3    12.5\n",
       "4    17.5\n",
       "5     NaN\n",
       "dtype: float64"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "numbers = np.linspace(0, 10, num=5) # makes numpy array([0, 2.5, 5, 7.5, 10])\n",
    "x = pd.Series(numbers) # index is [0, 1, 2, 3, 4]\n",
    "y = pd.Series(numbers, index=pd.Index([1, 2, 3, 4, 5]))\n",
    "x + y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We aren't limited to the integer indices of list-like structures, and we can label our rows. The labels can be altered at any time and be things like dates or even another column. In chapter 3, we will discuss how to perform some operations on the index in order to change it. Then, in chapter 4, we will use the index for operations merging data and aggregating it.\n",
    "\n",
    "## `DataFrame`\n",
    "Having a `Series` object for each column is an improvement over the NumPy representation; however, we still have the same problem when wanting to sort based on a value or grab an entire row out. The `DataFrame` gives us a representation of a table formed from many `Series` objects that form the columns and a shared `Index` object that labels the rows. We can create a `DataFrame` object from either of the NumPy representations we were working with earlier (we could also make a `Series` object for each column, but there is no need to do so):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>time</th>\n",
       "      <th>place</th>\n",
       "      <th>magType</th>\n",
       "      <th>mag</th>\n",
       "      <th>alert</th>\n",
       "      <th>tsunami</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2018-10-13 11:10:23.560</td>\n",
       "      <td>262km NW of Ozernovskiy, Russia</td>\n",
       "      <td>mww</td>\n",
       "      <td>6.7</td>\n",
       "      <td>green</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2018-10-13 04:34:15.580</td>\n",
       "      <td>25km E of Bitung, Indonesia</td>\n",
       "      <td>mww</td>\n",
       "      <td>5.2</td>\n",
       "      <td>green</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2018-10-13 00:13:46.220</td>\n",
       "      <td>42km WNW of Sola, Vanuatu</td>\n",
       "      <td>mww</td>\n",
       "      <td>5.7</td>\n",
       "      <td>green</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2018-10-12 21:09:49.240</td>\n",
       "      <td>13km E of Nueva Concepcion, Guatemala</td>\n",
       "      <td>mww</td>\n",
       "      <td>5.7</td>\n",
       "      <td>green</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2018-10-12 02:52:03.620</td>\n",
       "      <td>128km SE of Kimbe, Papua New Guinea</td>\n",
       "      <td>mww</td>\n",
       "      <td>5.6</td>\n",
       "      <td>green</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                      time                                  place magType  \\\n",
       "0  2018-10-13 11:10:23.560        262km NW of Ozernovskiy, Russia     mww   \n",
       "1  2018-10-13 04:34:15.580            25km E of Bitung, Indonesia     mww   \n",
       "2  2018-10-13 00:13:46.220              42km WNW of Sola, Vanuatu     mww   \n",
       "3  2018-10-12 21:09:49.240  13km E of Nueva Concepcion, Guatemala     mww   \n",
       "4  2018-10-12 02:52:03.620    128km SE of Kimbe, Papua New Guinea     mww   \n",
       "\n",
       "   mag  alert  tsunami  \n",
       "0  6.7  green        1  \n",
       "1  5.2  green        0  \n",
       "2  5.7  green        0  \n",
       "3  5.7  green        0  \n",
       "4  5.6  green        1  "
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.DataFrame(array_dict) \n",
    "\n",
    "# this will also work with the first representation\n",
    "# df = pd.DataFrame(data)\n",
    "\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can check the type of the underlying data with `dtypes` (note that it is not `dtype` as with `Series` and `Index` objects since each column will have its own data type):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "time        object\n",
       "place       object\n",
       "magType     object\n",
       "mag        float64\n",
       "alert       object\n",
       "tsunami      int64\n",
       "dtype: object"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can get the underlying data with the `values` attribute. Note that this looks very similar to our initial NumPy representation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([['2018-10-13 11:10:23.560', '262km NW of Ozernovskiy, Russia',\n",
       "        'mww', 6.7, 'green', 1],\n",
       "       ['2018-10-13 04:34:15.580', '25km E of Bitung, Indonesia', 'mww',\n",
       "        5.2, 'green', 0],\n",
       "       ['2018-10-13 00:13:46.220', '42km WNW of Sola, Vanuatu', 'mww',\n",
       "        5.7, 'green', 0],\n",
       "       ['2018-10-12 21:09:49.240',\n",
       "        '13km E of Nueva Concepcion, Guatemala', 'mww', 5.7, 'green', 0],\n",
       "       ['2018-10-12 02:52:03.620', '128km SE of Kimbe, Papua New Guinea',\n",
       "        'mww', 5.6, 'green', 1]], dtype=object)"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can isolate the columns with the `columns` attribute. Notice that the columns are actually an `Index` object just on a different axis (columns are the horizontal index while rows are the vertical index)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['time', 'place', 'magType', 'mag', 'alert', 'tsunami'], dtype='object')"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here are some commonly used attributes:\n",
    "\n",
    "|Attribute | Returns |\n",
    "| --- | --- |\n",
    "| `dtypes` | The data types of each column |\n",
    "| `shape` | Dimensions of the `DataFrame` object in a tuple of the form `(number of rows, number of columns)` |\n",
    "| `index` | The `Index` object along the rows of the `DataFrame` object |\n",
    "| `columns` | The name of the columns (as an `Index` object) |\n",
    "| `values` | The data in the `DataFrame` object |\n",
    "| `empty` | Check if the `DataFrame` object is empty |\n",
    "\n",
    "The `Index` object along the rows of the dataframe can be accessed via the `index` attribute (just as with `Series` objects):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RangeIndex(start=0, stop=5, step=1)"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.index"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As with both `Series` and `Index` objects, we can get the dimensions of the dataframe with the `shape` attribute. The result is of the form `(nrows, ncols)`. Our dataframe has 5 rows and 6 columns:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(5, 6)"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that we can also perform arithmetic on dataframes. Pandas will only perform the operation when both the index and column match. Here, we demonstrate addition. Since addition with strings means concatenation, `pandas` concatenated the string columns (`time`, `place`, `magType`, and `alert`) across dataframes. The numeric columns (`mag` and `tsunami`) were summed:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>time</th>\n",
       "      <th>place</th>\n",
       "      <th>magType</th>\n",
       "      <th>mag</th>\n",
       "      <th>alert</th>\n",
       "      <th>tsunami</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2018-10-13 11:10:23.5602018-10-13 11:10:23.560</td>\n",
       "      <td>262km NW of Ozernovskiy, Russia262km NW of Oze...</td>\n",
       "      <td>mwwmww</td>\n",
       "      <td>13.4</td>\n",
       "      <td>greengreen</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2018-10-13 04:34:15.5802018-10-13 04:34:15.580</td>\n",
       "      <td>25km E of Bitung, Indonesia25km E of Bitung, I...</td>\n",
       "      <td>mwwmww</td>\n",
       "      <td>10.4</td>\n",
       "      <td>greengreen</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2018-10-13 00:13:46.2202018-10-13 00:13:46.220</td>\n",
       "      <td>42km WNW of Sola, Vanuatu42km WNW of Sola, Van...</td>\n",
       "      <td>mwwmww</td>\n",
       "      <td>11.4</td>\n",
       "      <td>greengreen</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2018-10-12 21:09:49.2402018-10-12 21:09:49.240</td>\n",
       "      <td>13km E of Nueva Concepcion, Guatemala13km E of...</td>\n",
       "      <td>mwwmww</td>\n",
       "      <td>11.4</td>\n",
       "      <td>greengreen</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2018-10-12 02:52:03.6202018-10-12 02:52:03.620</td>\n",
       "      <td>128km SE of Kimbe, Papua New Guinea128km SE of...</td>\n",
       "      <td>mwwmww</td>\n",
       "      <td>11.2</td>\n",
       "      <td>greengreen</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                             time  \\\n",
       "0  2018-10-13 11:10:23.5602018-10-13 11:10:23.560   \n",
       "1  2018-10-13 04:34:15.5802018-10-13 04:34:15.580   \n",
       "2  2018-10-13 00:13:46.2202018-10-13 00:13:46.220   \n",
       "3  2018-10-12 21:09:49.2402018-10-12 21:09:49.240   \n",
       "4  2018-10-12 02:52:03.6202018-10-12 02:52:03.620   \n",
       "\n",
       "                                               place magType   mag  \\\n",
       "0  262km NW of Ozernovskiy, Russia262km NW of Oze...  mwwmww  13.4   \n",
       "1  25km E of Bitung, Indonesia25km E of Bitung, I...  mwwmww  10.4   \n",
       "2  42km WNW of Sola, Vanuatu42km WNW of Sola, Van...  mwwmww  11.4   \n",
       "3  13km E of Nueva Concepcion, Guatemala13km E of...  mwwmww  11.4   \n",
       "4  128km SE of Kimbe, Papua New Guinea128km SE of...  mwwmww  11.2   \n",
       "\n",
       "        alert  tsunami  \n",
       "0  greengreen        2  \n",
       "1  greengreen        0  \n",
       "2  greengreen        0  \n",
       "3  greengreen        0  \n",
       "4  greengreen        2  "
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df + df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr>\n",
    "<div>\n",
    "    <a href=\"../ch_01/introduction_to_data_analysis.ipynb\">\n",
    "        <button style=\"float: left;\">&#8592; Chapter 1</button>\n",
    "    </a>\n",
    "    <a href=\"./2-creating_dataframes.ipynb\">\n",
    "        <button style=\"float: right;\">Next Notebook &#8594;</button>\n",
    "    </a>\n",
    "</div>\n",
    "<br>\n",
    "<hr>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}