From b66b063b3cd8c85823afbc974a30480833c89717 Mon Sep 17 00:00:00 2001 From: chang-ning Date: Thu, 29 Nov 2018 11:31:05 +0800 Subject: [PATCH 1/6] add some notes in list --- docs/notes/python-list.rst | 187 +++++++++++++++++++++---------------- 1 file changed, 109 insertions(+), 78 deletions(-) diff --git a/docs/notes/python-list.rst b/docs/notes/python-list.rst index c4eeef58..68025cd4 100644 --- a/docs/notes/python-list.rst +++ b/docs/notes/python-list.rst @@ -11,30 +11,98 @@ some common operations and pitfalls. .. contents:: Table of Contents :backlinks: none +From Scratch +------------ -Get Items from a List ---------------------- - -Although getting data from a list is quite simple, we can retrieve data more -elegantly. Python provides a lot of ways to get data. Following example shows -how to get data via negative index and slice. +There are so many ways that we can manipulate lists in Python. Before we start +to learn those versatile manipulations, the following snippet shows the most +common operations of lists. .. code-block:: python >>> a = [1, 2, 3, 4, 5] - >>> a[0] - 1 + >>> # negative index >>> a[-1] 5 - >>> a[0:] - [1, 2, 3, 4, 5] - >>> a[:-1] - [1, 2, 3, 4] - >>> a[0:-1:2] # a[start:end:step] - [1, 3] + >>> # slicing list[start:end:step] + >>> a[1:] + [2, 3, 4, 5] + >>> a[1:-1] + [2, 3, 4] + >>> a[1:-1:2] + [2, 4] + >>> # reverse + >>> a[::-1] + [5, 4, 3, 2, 1] + >>> a[:0:-1] + [5, 4, 3, 2] + >>> # set an item + >>> a[0] = 0 + >>> a + [0, 2, 3, 4, 5] + >>> # append items to list + >>> a.append(6) + >>> a + [0, 2, 3, 4, 5, 6] + >>> a.extend([7, 8, 9]) + >>> a + [0, 2, 3, 4, 5, 6, 7, 8, 9] + >>> # delete an item + >>> del a[-1] + >>> a + [0, 2, 3, 4, 5, 6, 7, 8] + >>> # list comprehension + >>> b = [x for x in range(3)] + >>> b + [0, 1, 2] + >>> # add two lists + >>> a + b + [0, 2, 3, 4, 5, 6, 7, 8, 0, 1, 2] + +Using ``slice`` +--------------- + +Sometimes, our data may concatenate as a large segment such as packets. In +this case, we will represent the range of data by using ``slice`` objects +as explaining variables instead of using **slicing** expressions. + +.. code-block:: python + + >>> icmp = ( + ... b"080062988e2100005bff49c20005767c" + ... b"08090a0b0c0d0e0f1011121314151617" + ... b"18191a1b1c1d1e1f2021222324252627" + ... b"28292a2b2c2d2e2f3031323334353637" + ... ) + >>> head = slice(0, 32) + >>> data = slice(32, len(icmp)) + >>> icmp[head] + b'080062988e2100005bff49c20005767c' + +List Comprehensions +------------------- + +`List comprehensions `_ +which was proposed in PEP `202 `_ +provides a graceful way to create a new list based on another list, sequence, +or some object which is iterable. In addition, we can use this expression to +substitute ``map`` and ``filter`` sometimes. + +.. code-block:: python + + >>> [x for x in range(10)] + [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] + >>> [(lambda x: x**2)(i) for i in range(10)] + [0, 1, 4, 9, 16, 25, 36, 49, 64, 81] + >>> [x for x in range(10) if x > 5] + [6, 7, 8, 9] + >>> [x if x > 5 else 0 for x in range(10)] + [0, 0, 0, 0, 0, 0, 6, 7, 8, 9] + >>> [(x, y) for x in range(3) for y in range(2)] + [(0, 0), (0, 1), (1, 0), (1, 1), (2, 0), (2, 1)] -Unpacking a List ----------------- +Unpacking +--------- Sometimes, we want to unpack our list to variables in order to make our code become more readable. In this case, we assign N elements to N variables as @@ -60,11 +128,13 @@ than N in Python 3. >>> c [3, 4] -Get Index and Item ------------------- +Using ``enumerate`` +------------------- ``enumerate`` is a built-in function. It helps us to acquire indexes (or a count) and elements at the same time without using ``range(len(list))``. +Further information can be found on +`Looping Techniques `_. .. code-block:: python @@ -81,70 +151,31 @@ Get Index and Item 2 1 3 2 -Using a Slice -------------- - -Sometimes, our data may concatenate as a large segment. In this case, we will -represent the range of data by using ``slice`` objects as explaining variables -instead of using **slicing** expressions. - -.. code-block:: python - - - >>> record = "Hello world!" - >>> hello = slice(0, 5) # slice(start,end,step) - >>> world = slice(6, -1) - >>> record[hello] - 'Hello' - >>> record[world] - 'world' - -List Comprehensions -------------------- - -`List comprehensions `_ -which was proposed in PEP `202 `_ -provides a graceful way to create a new list based on another list, sequence, -or some object which is iterable. In addition, we can use this expression to -substitute ``map`` and ``filter`` sometimes. - -.. code-block:: python - - >>> [x for x in range(10)] - [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] - >>> [(lambda x: x**2)(i) for i in range(10)] - [0, 1, 4, 9, 16, 25, 36, 49, 64, 81] - >>> [x for x in range(10) if x > 5] - [6, 7, 8, 9] - >>> [x if x > 5 else 0 for x in range(10)] - [0, 0, 0, 0, 0, 0, 6, 7, 8, 9] - >>> [(x, y) for x in range(3) for y in range(2)] - [(0, 0), (0, 1), (1, 0), (1, 1), (2, 0), (2, 1)] +Zip Lists +--------- -Zip Multiple Lists ------------------- +``zip`` enables us to iterate over items contained in multiple lists at a time. +Iteration stops whenever one of the lists is exhausted. As a result, the +length of the iteration is the same as the shortest list. If this behavior is +not desired, we can use ``itertools.zip_longest`` in **Python 3** or +``itertools.izip_longest`` in **Python 2**. .. code-block:: python - >>> a = [1, 2, 3, 4, 5] - >>> b = [2, 4, 5, 6, 8] + >>> a = [1, 2, 3] + >>> b = [4, 5, 6] >>> list(zip(a, b)) - [(1, 2), (2, 4), (3, 5), (4, 6), (5, 8)] - >>> c = [5, 6, 7, 8] + [(1, 4), (2, 5), (3, 6)] + >>> c = [1] >>> list(zip(a, b, c)) - [(1, 2, 5), (2, 4, 6), (3, 5, 7), (4, 6, 8)] + [(1, 4, 1)] + >>> from itertools import zip_longest + >>> list(zip_longest(a, b, c)) + [(1, 4, 1), (2, 5, None), (3, 6, None)] -Reverse a List --------------- - -.. code-block:: python - - >>> a = [1, 2, 3, 4, 5] - >>> a[::-1] - [5, 4, 3, 2, 1] -Filter Unnecessary Items ------------------------- +Filter Items +------------ .. code-block:: python @@ -155,8 +186,8 @@ Filter Unnecessary Items >>> filter(predicate, l) [3, 4] -Using Lists as Stacks ---------------------- +Stacks +------ .. code-block:: python @@ -173,8 +204,8 @@ Using Lists as Stacks >>> stack [1] -Implement a List-like Object ----------------------------- +List-like +--------- .. code-block:: python From d9aac4116d1ef377bde4bb13e8a2c99c1f0f8a61 Mon Sep 17 00:00:00 2001 From: chang-ning Date: Thu, 29 Nov 2018 12:19:41 +0800 Subject: [PATCH 2/6] add initialize in list and change layout Signed-off-by: chang-ning --- docs/notes/python-list.rst | 78 +++++++++++++++++++++++--------------- 1 file changed, 47 insertions(+), 31 deletions(-) diff --git a/docs/notes/python-list.rst b/docs/notes/python-list.rst index 68025cd4..1e243e0b 100644 --- a/docs/notes/python-list.rst +++ b/docs/notes/python-list.rst @@ -59,6 +59,53 @@ common operations of lists. >>> a + b [0, 2, 3, 4, 5, 6, 7, 8, 0, 1, 2] +Initialize +---------- + +Generally speaking, we can create a list through ``*`` operator if the item in +the list expression is an immutable object. + +.. code-block:: python + + >>> >>> a = [None] * 3 + >>> a + [None, None, None] + >>> a[0] = "foo" + >>> a + ['foo', None, None] + +However, if the item in the list expression is a mutable object, the ``*`` +operator will copy the reference of the item N times. In order to avoid this +pitfall, we should use a list comprehension to initialize a list. + +.. code-block:: python + + >>> a = [[]] * 3 + >>> b = [[] for _ in range(3)] + >>> a[0].append("Hello") + >>> a + [['Hello'], ['Hello'], ['Hello']] + >>> b[0].append("Python") + >>> b + [['Python'], [], []] + +Copy +---- + +.. code-block:: python + + >>> a = [1, 2, 3] + >>> b = a + >>> a + [1, 2, 3] + >>> b + [1, 2, 3] + >>> b[2] = 123456 # a[2] = 123456 + >>> b + [1, 2, 123456] + >>> a + [1, 2, 123456] + Using ``slice`` --------------- @@ -251,34 +298,3 @@ List-like EmuList: [0, 1, 3, 6, 5] >>> 0 in emul # __contains__ True - -Pitfall - Assign a List to Another Variable -------------------------------------------- - -.. code-block:: python - - >>> a = [1, 2, 3] - >>> b = a - >>> a - [1, 2, 3] - >>> b - [1, 2, 3] - >>> b[2] = 123456 # a[2] = 123456 - >>> b - [1, 2, 123456] - >>> a - [1, 2, 123456] - -Pitfall - Init a List with Multiply ------------------------------------ - -.. code-block:: python - - >>> a = [[]] * 3 - >>> b = [[] for _ in range(3)] - >>> a[0].append("Hello") - >>> a - [['Hello'], ['Hello'], ['Hello']] - >>> b[0].append("Python") - >>> b - [['Python'], [], []] From 466dca9d9524258a818fb88bba301877e48a92f9 Mon Sep 17 00:00:00 2001 From: chang-ning Date: Thu, 29 Nov 2018 18:40:12 +0800 Subject: [PATCH 3/6] add some descriptions in list Signed-off-by: chang-ning --- docs/notes/python-list.rst | 29 ++++++++++++++++++++--------- 1 file changed, 20 insertions(+), 9 deletions(-) diff --git a/docs/notes/python-list.rst b/docs/notes/python-list.rst index 1e243e0b..e838542b 100644 --- a/docs/notes/python-list.rst +++ b/docs/notes/python-list.rst @@ -67,7 +67,7 @@ the list expression is an immutable object. .. code-block:: python - >>> >>> a = [None] * 3 + >>> a = [None] * 3 >>> a [None, None, None] >>> a[0] = "foo" @@ -111,7 +111,7 @@ Using ``slice`` Sometimes, our data may concatenate as a large segment such as packets. In this case, we will represent the range of data by using ``slice`` objects -as explaining variables instead of using **slicing** expressions. +as explaining variables instead of using *slicing expressions*. .. code-block:: python @@ -201,11 +201,12 @@ Further information can be found on Zip Lists --------- -``zip`` enables us to iterate over items contained in multiple lists at a time. -Iteration stops whenever one of the lists is exhausted. As a result, the -length of the iteration is the same as the shortest list. If this behavior is -not desired, we can use ``itertools.zip_longest`` in **Python 3** or -``itertools.izip_longest`` in **Python 2**. +`zip `_ enables us to +iterate over items contained in multiple lists at a time. Iteration stops +whenever one of the lists is exhausted. As a result, the length of the +iteration is the same as the shortest list. If this behavior is not desired, +we can use ``itertools.zip_longest`` in **Python 3** or ``itertools.izip_longest`` +in **Python 2**. .. code-block:: python @@ -224,13 +225,23 @@ not desired, we can use ``itertools.zip_longest`` in **Python 3** or Filter Items ------------ +`filter `_ is a +built-in function to assist us to remove unnecessary items. In **Python 2**, +``filter`` returns a list. However, in **Python 3**, ``filter`` returns an +*iterable object*. Note that *list comprehension* or *generator +expression* provides a more concise way to remove items. + .. code-block:: python >>> [x for x in range(5) if x > 1] [2, 3, 4] >>> l = ['1', '2', 3, 'Hello', 4] - >>> predicate = lambda x: isinstance(x, int) - >>> filter(predicate, l) + >>> f = lambda x: isinstance(x, int) + >>> filter(f, l) + + >>> list(filter(f, l)) + [3, 4] + >>> list((i for i in l if f(i))) [3, 4] Stacks From bc35d32d0469dc06c3d77e31eddd562e5f7b2616 Mon Sep 17 00:00:00 2001 From: chang-ning Date: Thu, 29 Nov 2018 22:09:53 +0800 Subject: [PATCH 4/6] add copy in list Signed-off-by: chang-ning --- docs/notes/python-list.rst | 45 ++++++++++++++++++++++++++++++++++---- 1 file changed, 41 insertions(+), 4 deletions(-) diff --git a/docs/notes/python-list.rst b/docs/notes/python-list.rst index e838542b..174285d8 100644 --- a/docs/notes/python-list.rst +++ b/docs/notes/python-list.rst @@ -92,20 +92,57 @@ pitfall, we should use a list comprehension to initialize a list. Copy ---- +Assigning a list to a variable is a common pitfall. This assignment does not +copy the list to the variable. The variable only refers to the list and increase +the reference count of the list. + .. code-block:: python + import sys >>> a = [1, 2, 3] + >>> sys.getrefcount(a) + 2 >>> b = a - >>> a - [1, 2, 3] - >>> b - [1, 2, 3] + >>> sys.getrefcount(a) + 3 >>> b[2] = 123456 # a[2] = 123456 >>> b [1, 2, 123456] >>> a [1, 2, 123456] +There are two types of copy. The first one is called *shallow copy* (non-recursive copy) +and the second one is called *deep copy* (recursive copy). Most of the time, it +is sufficient for us to copy a list by shallow copy. However, if a list is nested, +we have to use a deep copy. + +.. code-block:: python + + >>> # shallow copy + >>> a = [1, 2] + >>> b = list(a) + >>> b[0] = 123 + >>> a + [1, 2] + >>> b + [123, 2] + >>> a = [[1], [2]] + >>> b = list(a) + >>> b[0][0] = 123 + >>> a + [[123], [2]] + >>> b + [[123], [2]] + >>> # deep copy + >>> import copy + >>> a = [[1], [2]] + >>> b = copy.deepcopy(a) + >>> b[0][0] = 123 + >>> a + [[1], [2]] + >>> b + [[123], [2]] + Using ``slice`` --------------- From 37da061636bfdeb1098d8b3542fcd3c0ca60a454 Mon Sep 17 00:00:00 2001 From: chang-ning Date: Thu, 29 Nov 2018 23:20:29 +0800 Subject: [PATCH 5/6] refactor list magic methods --- docs/notes/python-list.rst | 156 +++++++++++++++++++++++++++---------- 1 file changed, 113 insertions(+), 43 deletions(-) diff --git a/docs/notes/python-list.rst b/docs/notes/python-list.rst index 174285d8..d1eafb37 100644 --- a/docs/notes/python-list.rst +++ b/docs/notes/python-list.rst @@ -21,6 +21,12 @@ common operations of lists. .. code-block:: python >>> a = [1, 2, 3, 4, 5] + >>> # contains + >>> 2 in a + True + >>> # positive index + >>> a[0] + 1 >>> # negative index >>> a[-1] 5 @@ -299,50 +305,114 @@ Stacks >>> stack [1] -List-like ---------- +Elements ``in`` List-like +------------------------- .. code-block:: python - >>> class EmuList(object): - ... def __init__(self, list_): - ... self._list = list_ - ... def __repr__(self): - ... return "EmuList: " + repr(self._list) - ... def append(self, item): - ... self._list.append(item) - ... def remove(self, item): - ... self._list.remove(item) - ... def __len__(self): - ... return len(self._list) - ... def __getitem__(self, sliced): - ... return self._list[sliced] - ... def __setitem__(self, sliced, val): - ... self._list[sliced] = val - ... def __delitem__(self, sliced): - ... del self._list[sliced] - ... def __contains__(self, item): - ... return item in self._list - ... def __iter__(self): - ... return iter(self._list) - ... - >>> emul = EmuList(range(5)) - >>> emul - EmuList: [0, 1, 2, 3, 4] - >>> emul[1:3] # __getitem__ - [1, 2] - >>> emul[0:4:2] # __getitem__ - [0, 2] - >>> len(emul) # __len__ - 5 - >>> emul.append(5) - >>> emul - EmuList: [0, 1, 2, 3, 4, 5] - >>> emul.remove(2) - >>> emul - EmuList: [0, 1, 3, 4, 5] - >>> emul[3] = 6 # __setitem__ - >>> emul - EmuList: [0, 1, 3, 6, 5] - >>> 0 in emul # __contains__ + class Stack: + + def __init__(self): + self.__list = [] + + def push(self, val): + self.__list.append(val) + + def pop(self): + return self.__list.pop() + + def __contains__(self, item): + return True if item in self.__list else False + + stack = Stack() + stack.push(1) + print(1 in stack) + print(0 in stack) + +Example + +.. code-block:: bash + + python stack.py True + False + +Accessing List-like Items +------------------------- + +.. code-block:: python + + class Stack: + + def __init__(self): + self.__list = [] + + def push(self, val): + self.__list.append(val) + + def pop(self): + return self.__list.pop() + + def __repr__(self): + return "{}".format(self.__list) + + def __len__(self): + return len(self.__list) + + def __getitem__(self, idx): + return self.__list[idx] + + def __setitem__(self, idx, val): + self.__list[idx] = val + + + stack = Stack() + stack.push(1) + stack.push(2) + print("stack:", stack) + + stack[0] = 3 + print("stack:", stack) + print("num items:", len(stack)) + +Example + +.. code-block:: bash + + $ python stack.py + stack: [1, 2] + stack: [3, 2] + num items: 2 + +Delegating List-like Iteration +------------------------------ + +.. code-block:: python + + class Stack: + + def __init__(self): + self.__list = [] + + def push(self, val): + self.__list.append(val) + + def pop(self): + return self.__list.pop() + + def __iter__(self): + return iter(self.__list) + + stack = Stack() + stack.push(1) + stack.push(2) + for s in stack: + print(s) + +Example + +.. code-block:: bash + + $ python stack.py + 1 + 2 From 850803083338be7a69f8244f81d1a5570cb0c93d Mon Sep 17 00:00:00 2001 From: chang-ning Date: Fri, 30 Nov 2018 07:48:54 +0800 Subject: [PATCH 6/6] add some descriptions in list Signed-off-by: chang-ning --- docs/notes/python-list.rst | 31 +++++++++++++++++++++++++------ 1 file changed, 25 insertions(+), 6 deletions(-) diff --git a/docs/notes/python-list.rst b/docs/notes/python-list.rst index d1eafb37..419c01f5 100644 --- a/docs/notes/python-list.rst +++ b/docs/notes/python-list.rst @@ -290,6 +290,10 @@ expression* provides a more concise way to remove items. Stacks ------ +There is no need an additional data structure, stack, in Python because the +``list`` provides ``append`` and ``pop`` methods which enable us use a list as +a stack. + .. code-block:: python >>> stack = [] @@ -305,8 +309,12 @@ Stacks >>> stack [1] -Elements ``in`` List-like -------------------------- +``in`` Operation +---------------- + +We can implement the ``__contains__`` method to make a class do ``in`` +operations. It is a common way for a programmer to emulate +a membership test operations for custom classes. .. code-block:: python @@ -337,8 +345,14 @@ Example True False -Accessing List-like Items -------------------------- +Accessing Items +--------------- + +Making custom classes perform get and set operations like lists is simple. We +can implement a ``__getitem__`` method and a ``__setitem__`` method to enable +a class to retrieve and overwrite data by index. In addition, if we want to use +the function, ``len``, to calculate the number of elements, we can implement a +``__len__`` method. .. code-block:: python @@ -384,8 +398,13 @@ Example stack: [3, 2] num items: 2 -Delegating List-like Iteration ------------------------------- +Delegating Iterations +--------------------- + +If a custom container class holds a list and we want iteration work in the +container, we can implement a ``__iter__`` method to delegate iterations to +the list. Note that the method, ``__iter__`` should return an *iterator object*, +so we cannot return the list directly; otherwise, Python raises a ``TypeError``. .. code-block:: python