generator_example.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "2de55b12",
   "metadata": {},
   "source": [
    "## Generator example\n",
    "    Generator를 쓰면 메모리 용량을 아낄 수 있다.\n",
    "    특히, 대용량의 데이터인 경우 Generator를 쓰는게 필수다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "aeaace52",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-04-20T16:34:36.983499Z",
     "start_time": "2022-04-20T16:34:36.961676Z"
    }
   },
   "outputs": [],
   "source": [
    "import sys"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "dbe15e9e",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-04-20T16:34:37.148762Z",
     "start_time": "2022-04-20T16:34:37.141338Z"
    }
   },
   "outputs": [],
   "source": [
    "def general_list(n):\n",
    "    return [i for i in range(n)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "8f585960",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-04-20T16:34:37.350293Z",
     "start_time": "2022-04-20T16:34:37.342248Z"
    }
   },
   "outputs": [],
   "source": [
    "def generator_list(n):\n",
    "    for i in range(n):\n",
    "        yield i"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "61ad830d",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-04-20T16:34:37.560019Z",
     "start_time": "2022-04-20T16:34:37.549427Z"
    }
   },
   "outputs": [],
   "source": [
    "N=100000"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "39ed9c66",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-04-20T16:34:37.777620Z",
     "start_time": "2022-04-20T16:34:37.765010Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49] 100000\n",
      "824456\n"
     ]
    }
   ],
   "source": [
    "result = general_list(N)\n",
    "print(result[:50], len(result))\n",
    "print(sys.getsizeof(result))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "e4d9b978",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-04-20T16:35:03.694459Z",
     "start_time": "2022-04-20T16:35:03.685166Z"
    }
   },
   "outputs": [],
   "source": [
    "result2 = generator_list(N)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "00d1fe95",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-04-20T16:35:02.389389Z",
     "start_time": "2022-04-20T16:35:02.382400Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "112\n"
     ]
    }
   ],
   "source": [
    "print(sys.getsizeof(result2))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "1cd2f149",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-04-20T16:35:02.670767Z",
     "start_time": "2022-04-20T16:35:02.640848Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49]\n"
     ]
    }
   ],
   "source": [
    "print(list(result2)[:50])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "9523838e",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-04-20T16:35:04.988297Z",
     "start_time": "2022-04-20T16:35:04.970236Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "100000\n"
     ]
    }
   ],
   "source": [
    "print(len(list(result2)))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "95dca759",
   "metadata": {},
   "source": [
    "## Generator comprehension\n",
    "    list comprehension과 유사한 형태로 generator형태의 list 생성\n",
    "    generator expression 이라고도 함\n",
    "    [] 대신 () 를 사용하여 표현\n",
    "    사용하려면 for loop에 넣거나 list에 넣어 사용가능"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "8612436a",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-04-20T16:57:41.008429Z",
     "start_time": "2022-04-20T16:57:41.000124Z"
    }
   },
   "outputs": [],
   "source": [
    "gen_ex = (n*n for n in range(500))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "f5a5a54e",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-04-20T16:55:34.195816Z",
     "start_time": "2022-04-20T16:55:34.187702Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'generator'>\n"
     ]
    }
   ],
   "source": [
    "print(type(gen_ex))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "a06c1912",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-04-20T16:57:14.160809Z",
     "start_time": "2022-04-20T16:57:14.153526Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "112\n"
     ]
    }
   ],
   "source": [
    "print(sys.getsizeof(gen_ex))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "3ca6628a",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-04-20T16:57:42.871741Z",
     "start_time": "2022-04-20T16:57:42.859435Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]\n"
     ]
    }
   ],
   "source": [
    "print(list(gen_ex)[:10])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c5bf7bae",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-04-20T16:59:34.682301Z",
     "start_time": "2022-04-20T16:59:34.673166Z"
    }
   },
   "source": [
    "### Generator를 사용하는 경우\n",
    "    - 대용량 파일 데이터를 처리할 때\n",
    "    - list 타입 데이터를 반환해주는 함수 만들 때\n",
    "    -> 중간 과정에 loop이 중단될 수 있을 때도 유용\n",
    "    - 큰 데이터 처리할 때는 generator comprehension(expression)을 고려 \n",
    "    -> 메모리 주소만 할당해서 데이터가 커도 처리에 어려움없다"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.12"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}