Python Sequences¶
Ordered collections of objects
What is a Sequence?¶
A sequence is an ordered collection of objects.
They are analogous to what are often called “arrays” or “lists” in other programming languages.
But in Python, there are number of types that all fit this description, each with special customization. But any object that has the behavior expected of a sequence can be treated the same way in Python:
Remember Duck Typing?
If it looks like a duck and quacks like a duck…
OR: If it looks and acts like a sequence – it is a sequence.
Technically, if it satisfies the “Sequence Protocol”, it is a sequence.
Python is all about these protocols – we will see more of them.
The Sequence Protocol¶
A sequence can be considered as anything that supports at least these operations:
- Indexing
- Slicing
- Membership
- Concatenation
- Length
- Iteration
I’ll get into all of those as we go along.
Sequence Types¶
There are eight built in types in Python that are sequences:
- string
- list
- tuple
- bytes
- bytearray
- buffer
- array.array
- range object (almost)
For this lesson, you won’t see much beyond strings, lists, and tuples – the rest are pretty special purpose.
But what we learn in this lesson applies to all sequences (with minor caveats).
I’ll use lists, strings and tuples in the examples.
So let’s take a look at the key parts of the sequence protocol:
Indexing¶
Items in a sequence may be looked up by index using the indexing
operator: []
Indexing in Python always starts at zero.
Here is an example with a string – a string is a sequence of characters.
In [98]: s = "this is a string"
In [99]: s[0]
Out[99]: 't'
In [100]: s[5]
Out[100]: 'i'
Note that the first character is indexed with zero – I sometimes call that the “zeroth” item in the sequence.
Zero indexing may seem odd at first (if you are not already a programming geek), but it turns out to make a lot of things easier. More on that later.
You can use negative indexes to count from the end:
In [2]: a_list = [34, 56, 19, 23, 55]
In [3]: a_list[-1]
Out[3]: 55
In [4]: a_list[-2]
Out[4]: 23
In [5]: a_list[-4]
Out[5]: 56
Indexing beyond the end of a sequence causes an IndexError:
In [6]: a_list
Out[6]: [34, 56, 19, 23, 55]
In [7]: a_list[5]
---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
<ipython-input-7-c1f9ac3b6fee> in <module>()
----> 1 a_list[5]
IndexError: list index out of range
Pretty straight forward so far…
Slicing¶
Slicing is a real “power tool” of Python – it can allow very short code.
Slicing a sequence creates a new sequence with a range of objects from the original sequence.
It also uses the indexing operator ([]
), but with a twist.
sequence[start:finish]
returns all sequence[i] for which start <= i < finish
That’s a fancy way to say that it’s all the items from start to finish – including start, but NOT including finish.
This also may be a bit unintuitive – but it’s very practical.
In [121]: s = "a bunch of words"
In [122]: s[2]
Out[122]: 'b'
In [123]: s[6]
Out[123]: 'h'
In [124]: s[2:6]
Out[124]: 'bunc'
In [125]: s[2:7]
Out[125]: 'bunch'
Helpful Hint¶
It can really help if you think about slicing this way:
(write this out!)
Think of the indexes as pointing to the spaces between the items:
a b u n c h o f w o r d s
| | | | | | | | | | | | | | | |
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Slicing¶
Python has some other slicing shortcuts…
You do not have to provide both start
and finish
:
In [6]: s = "a bunch of words"
In [7]: s[:5]
Out[7]: 'a bun'
In [8]: s[5:]
Out[8]: 'ch of words'
Either 0
or len(s)
will be assumed, respectively.
You can combine this with the negative index to get the end of a sequence:
In [4]: s = 'this_could_be_a_filename.txt'
In [5]: s[:-4]
Out[5]: 'this_could_be_a_filename'
In [6]: s[-4:]
Out[6]: '.txt'
That is a real-world example I use all the time.
Why start from zero?¶
Python indexing feels ‘weird’ to some folks – particularly those that don’t come with a background in the C family of languages.
Why is the “first” item indexed with zero?
Why is the last item in the slice not included?
Because these lead to some nifty properties:
len(seq[a:b]) == b-a
seq[:b] + seq[b:] == seq
len(seq[:b]) == b
len(seq[-b:]) == b
There are very many fewer “off by one” errors as a result.
More on Slicing¶
Slicing takes a third argument: step
which controls which items are
returned:
In [18]: a_tuple
Out[18]: (0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19)
In [19]: a_tuple[0:15]
Out[19]: (0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14)
In [20]: a_tuple[0:15:2]
Out[20]: (0, 2, 4, 6, 8, 10, 12, 14)
In [21]: a_tuple[0:15:3]
Out[21]: (0, 3, 6, 9, 12)
In [22]: a_tuple[::-1]
Out[22]: (19, 18, 17, 16, 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0)
Very cool – a negative step reverses the results!
Slicing vs. Indexing¶
Though they share an operator, slicing and indexing have a few important differences:
- Indexing will always return one single object (a scalar), whereas slicing will return a sequence of objects.
So if you start with, say, a list of numbers, indexing will return a single number. Slicing, on the other hand, will return list of numbers – even if that list only has one number in it – or zero!
Note that strings are a bit of an exception – there is no character type in Python – so a single character is a string – a sequence of length-1.
- Indexing past the end of a sequence will raise an error, slicing will not:
In [129]: s = "a bunch of words"
In [130]: s[17]
----> 1 s[17]
IndexError: string index out of range
In [131]: s[10:20]
Out[131]: ' words'
In [132]: s[20:30]
Out[132]: ''
(try it yourself….)
Membership¶
All sequences support the in
and not in
membership operators:
In [15]: s = [1, 2, 3, 4, 5, 6]
In [16]: 5 in s
Out[16]: True
In [17]: 42 in s
Out[17]: False
In [18]: 42 not in s
Out[18]: True
For strings, the membership operations are like substring
operations in
other languages:
In [20]: s = "This is a long string"
In [21]: "long" in s
Out[21]: True
This does not work for sub-sequences of other types (can you think of why?):
In [22]: s = [1, 2, 3, 4]
In [23]: [2, 3] in s
Out[23]: False
Concatenation¶
Using +
or *
on sequences will concatenate them:
In [18]: l1 = [1,2,3,4]
In [19]: l2 = [5,6,7,8]
In [20]: l1 + l2
Out[20]: [1, 2, 3, 4, 5, 6, 7, 8]
In [21]: (l1+l2) * 2
Out[21]: [1, 2, 3, 4, 5, 6, 7, 8, 1, 2, 3, 4, 5, 6, 7, 8]
Multiplying and Slicing¶
You can apply this concatenation to slices as well, leading to some nicely concise code:
from CodingBat: Warmup-1 – front3
def front3(str):
if len(str) < 3:
return str+str+str
else:
return str[:3]+str[:3]+str[:3]
This non-pythonic solution can also be expressed like so:
def front3(str):
return str[:3] * 3
Length¶
All sequences have a length. You can get it with the len
builtin:
In [36]: s = "how long is this, anyway?"
In [37]: len(s)
Out[37]: 25
Remember: Sequences are 0-indexed, so the last index is len(s)-1
:
In [38]: count = len(s)
In [39]: s[count]
------------------------------------------------------------
IndexError Traceback (most recent call last)
<ipython-input-39-5a33b9d3e525> in <module>()
----> 1 s[count]
IndexError: string index out of range
Better to use s[-1]
Miscellaneous¶
There are a bunch more operations supported by most sequences. Min and Max ———–
All sequences also support the min
and max
builtins:
In [42]: all_letters = "thequickbrownfoxjumpedoverthelazydog"
In [43]: min(all_letters)
Out[43]: 'a'
In [44]: max(all_letters)
Out[44]: 'z'
Why are those the answers you get? (hint: ord('a')
)
Of course this works with numbers, too!
In [1]: seq = [4,2,8,3,5,8,5,7]
In [2]: min(seq)
Out[2]: 2
In [3]: max(seq)
Out[3]: 8
Index¶
All sequences also support the index
method, which returns the index of the first occurrence of an item in the sequence:
In [46]: all_letters.index('d')
Out[46]: 21
This causes a ValueError
if the item is not in the sequence:
In [47]: all_letters.index('A')
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-47-2db728a46f78> in <module>()
----> 1 all_letters.index('A')
ValueError: substring not found
Count¶
A sequence can also be queried for the number of times a particular item appears:
In [52]: all_letters.count('o')
Out[52]: 4
In [53]: all_letters.count('the')
Out[53]: 2
This does not raise an error if the item you seek is not present:
In [54]: all_letters.count('A')
Out[54]: 0
Iteration¶
All sequences are “iterables”.
You can iterate over a sequence with for
:
for element in sequence:
do_something(element)
Which is what we mean when we say a sequence is an “iterable”.
There are some complexities about that – but more on that in another lesson.
Lists, Tuples…¶
The primary sequence types.
Lists¶
Lists can be constructed using list literals ([]
):
In [1]: []
Out[1]: []
In [2]: [1,2,3]
Out[2]: [1, 2, 3]
In [3]: [1, 'a', 7.34]
Out[3]: [1, 'a', 7.34]
Or by using the list
type object as a constructor:
In [6]: list()
Out[6]: []
In [7]: list(range(4))
Out[7]: [0, 1, 2, 3]
In [8]: list('abc')
Out[8]: ['a', 'b', 'c']
It will take any “iterable” (which means any sequence automatically – remember that all sequences are iterable?)
List Elements¶
The elements contained in a list need not be of a single type.
Lists are heterogenous, ordered collections.
Each element in a list is a value, and can be in multiple lists and have multiple names (or no name):
In [9]: name = 'Brian'
In [10]: a = [1, 2, name]
In [11]: b = [3, 4, name]
In [12]: a[2]
Out[12]: 'Brian'
In [13]: b[2]
Out[13]: 'Brian'
In [14]: a[2] is b[2]
Out[14]: True
Notice that even with a “literal” – the elements don’t need to be literals as well – they can be names.
They can even be function calls:
In [4]: def fun(n):
...: return n * 2
...:
In [5]: l = [3, 'four', fun(3), fun(9)]
In [6]: l
Out[6]: [3, 'four', 6, 18]
Tuples¶
Tuples can be constructed using tuple literals (()
):
In [15]: ()
Out[15]: ()
In [16]: (1, 2)
Out[16]: (1, 2)
In [17]: (1, 'a', 7.65)
Out[17]: (1, 'a', 7.65)
In [18]: (1,)
Out[18]: (1,)
Tuples and Commas…¶
Tuples don’t NEED parentheses…
In [161]: t = (1,2,3)
In [162]: t
Out[162]: (1, 2, 3)
In [163]: t = 1,2,3
In [164]: t
Out[164]: (1, 2, 3)
In [165]: type(t)
Out[165]: tuple
But they do need commas…!
In [156]: t = ( 3 )
In [157]: type(t)
Out[157]: int
In [158]: t = ( 3, )
In [160]: type(t)
Out[160]: tuple
This is a Python “gotcha” – some folks on my team recently had a weird bug that two of them could not figure out. They were getting a type error – something like:
TypeError: unsupported operand type(s) for /: ‘tuple’ and ‘float’
which made no sense – there were no tuples involved – in this case, the value was being pulled from a list – and it WAS a float. They even put type checking code in there, and it was, indeed, a float.
After poking at the code a bit, I suddenly spotted an extra comma – BINGO! that was it.
The code was more involved, and thus harder to see, but it was pretty much like this:
In [16]: l = [3, 4, 5, 6]
In [17]: x = l[3],
then a bit further down, x was used:
In [18]: y = x / 2.0
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-18-5289811a13ac> in <module>()
----> 1 y = x / 2.0
TypeError: unsupported operand type(s) for /: 'tuple' and 'float'
Would you have seen that?
Converting something to a Tuple¶
You can also use the tuple
type object to convert any iterable (sequence) into a tuple:
In [20]: tuple()
Out[20]: ()
In [21]: tuple(range(4))
Out[21]: (0, 1, 2, 3)
In [22]: tuple('garbanzo')
Out[22]: ('g', 'a', 'r', 'b', 'a', 'n', 'z', 'o')
Tuple Elements¶
The elements contained in a tuple need not be of a single type.
Tuples are heterogenous, ordered collections.
Each element in a tuple is a value, and can be in multiple tuples and have multiple names (or no name):
In [23]: name = 'Brian'
In [24]: other = name
In [25]: a = (1, 2, name)
In [26]: b = (3, 4, other)
In [27]: for i in range(3):
....: print(a[i] is b[i], end=' ')
....:
False False True
Look familiar from lists??
Lists vs. Tuples¶
So why have both?
Mutability in Python¶
All objects in Python fall into one of two camps:
- Mutable
- Immutable
Objects which are mutable may be changed in place.
Objects which are immutable may not be changed.
Ever.
The Types We Know¶
Immutable | Mutable |
---|---|
String | List |
Integer | Dictionary |
Float | |
Tuple |
This may make it look like the Mutables are rare – but in fact, most “container types”, and most custom objects are mutable.
Immutable types are the exception
Lists Are Mutable¶
Try this out:
In [28]: food = ['spam', 'eggs', 'ham']
In [29]: food
Out[29]: ['spam', 'eggs', 'ham']
In [30]: food[1] = 'raspberries'
In [31]: food
Out[31]: ['spam', 'raspberries', 'ham']
We repeat the exercise with a Tuple:
In [32]: food = ('spam', 'eggs', 'ham')
In [33]: food
Out[33]: ('spam', 'eggs', 'ham')
In [34]: food[1] = 'raspberries'
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-34-0c3401794933> in <module>()
----> 1 food[1] = 'raspberries'
TypeError: 'tuple' object does not support item assignment
Watch Out when name binding¶
This property means you need to be aware of what you are doing with your lists:
In [36]: original = [1, 2, 3]
In [37]: altered = original
In [38]: for i in range(len(original)):
....: if True:
....: altered[i] += 1
....:
Perhaps we want to check to see if altered has been updated, as a flag for whatever condition caused it to be updated.
What is the result of this code?
Perhaps Not What You Expect¶
Our altered
list has been updated as we’d expect:
In [39]: altered
Out[39]: [2, 3, 4]
But so has the original
list:
In [40]: original
Out[40]: [2, 3, 4]
Why?
Let’s look at that code again.
What does the line: altered = original
do?
It binds the name: “altered” to the same object that “original” is bound to.
That is, there is only one list, even though it is referred to by two names. So when you mutate (or change) that list from either name, the changes show up when you refer to it by the other name.
Other Gotchas¶
Easy container setup, or deadly trap?
(note: you can nest lists to make a 2D-ish array)
In [13]: bins = [ [] ] * 5
In [14]: bins
Out[14]: [[], [], [], [], []]
In [15]: words = ['one', 'three', 'rough', 'sad', 'goof']
In [16]: for word in words:
....: bins[len(word)-1].append(word)
....:
So, what is going to be in bins
now?
There is only One bin¶
In [65]: bins
Out[65]:
[['one', 'three', 'rough', 'sad', 'goof'],
['one', 'three', 'rough', 'sad', 'goof'],
['one', 'three', 'rough', 'sad', 'goof'],
['one', 'three', 'rough', 'sad', 'goof'],
['one', 'three', 'rough', 'sad', 'goof']]
We multiplied a sequence containing a single mutable object.
We got a list containing five references to a single mutable object.
Mutable Default Argument¶
Watch out especially for passing mutable objects as default values for function parameters:
In [71]: def accumulator(count, ac_list=[]):
....: for i in range(count):
....: ac_list.append(i)
....: return ac_list
....:
In [72]: accumulator(5)
Out[72]: [0, 1, 2, 3, 4]
In [73]: accumulator(7)
Out[73]: [0, 1, 2, 3, 4, 0, 1, 2, 3, 4, 5, 6]
What is going on here???
It turns out that that code: ac_list=[]
is evaluated when the function is defined – not when the function is called.
So the name “ac_list” in the local scope of that function always refers to the same list. So every time the function is called, more is added to that same list.
The moral of the story here is:
Do not use mutable objects for default arguments!
It turns out that this early evaluation can be useful – but for now, just remember not to use mutables as default arguments.
By the way –this is how you should write that code:
In [21]: def accumulator(count, ac_list=None):
...: if ac_list is None:
...: ac_list = []
...: for i in range(count):
...: ac_list.append(i)
...: return ac_list
In [22]: accumulator(5)
Out[22]: [0, 1, 2, 3, 4]
In [23]: accumulator(7)
Out[23]: [0, 1, 2, 3, 4, 5, 6]
This will ensure that a new list will be created if one is not passed-in.
Mutable Sequence Methods¶
In addition to all the methods supported by sequences we’ve seen above, mutable sequences (the List), have a number of other methods that are used to change it in place.
You can find all these in the Standard Library Documentation:
https://docs.python.org/3/library/stdtypes.html#typesseq-mutable
Assignment¶
You’ve already seen changing a single element of a list by assignment.
Pretty much the same as “arrays” in most languages:
In [100]: my_list = [1, 2, 3]
In [101]: my_list[2] = 10
In [102]: my_list
Out[102]: [1, 2, 10]
Growing the List¶
.append()
, .insert()
, .extend()
In [74]: food = ['spam', 'eggs', 'ham']
In [75]: food.append('sushi')
In [76]: food
Out[76]: ['spam', 'eggs', 'ham', 'sushi']
In [77]: food.insert(0, 'beans')
In [78]: food
Out[78]: ['beans', 'spam', 'eggs', 'ham', 'sushi']
In [79]: food.extend(['bread', 'water'])
In [80]: food
Out[80]: ['beans', 'spam', 'eggs', 'ham', 'sushi', 'bread', 'water']
More on Extend¶
You can pass any sequence to .extend()
:
In [85]: food
Out[85]: ['beans', 'spam', 'eggs', 'ham', 'sushi', 'bread', 'water']
In [86]: food.extend('spaghetti')
In [87]: food
Out[87]:
['beans', 'spam', 'eggs', 'ham', 'sushi', 'bread', 'water',
's', 'p', 'a', 'g', 'h', 'e', 't', 't', 'i']
So be careful – a string is a single object –but also a sequence of characters.
Shrinking the List¶
.pop()
, .remove()
In [203]: food = ['spam', 'eggs', 'ham', 'toast']
In [204]: food.pop()
Out[204]: 'toast'
In [205]: food.pop(0)
Out[205]: 'spam'
In [206]: food
Out[206]: ['eggs', 'ham']
In [207]: food.remove('ham')
In [208]: food
Out[208]: ['eggs']
You can also delete slices of a list with the del
keyword:
In [92]: nums = range(10)
In [93]: nums
Out[93]: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
In [94]: del nums[1:6:2]
In [95]: nums
Out[95]: [0, 2, 4, 6, 7, 8, 9]
In [96]: del nums[-3:]
In [97]: nums
Out[97]: [0, 2, 4, 6]
Copying Lists¶
You can make copies of part of a list using slicing:
In [227]: food = ['spam', 'eggs', 'ham', 'sushi']
In [228]: some_food = food[1:3]
In [229]: some_food[1] = 'bacon'
In [230]: food
Out[230]: ['spam', 'eggs', 'ham', 'sushi']
In [231]: some_food
Out[231]: ['eggs', 'bacon']
If you provide no arguments to the slice, it makes a copy of the entire list:
In [232]: food
Out[232]: ['spam', 'eggs', 'ham', 'sushi']
In [233]: food2 = food[:]
In [234]: food is food2
Out[234]: False
Shallow Copies¶
The copy of a list made this way is a shallow copy.
The list is itself a new object, but the objects it contains are not.
Mutable objects in the list can be mutated in both copies:
In [249]: food = ['spam', ['eggs', 'ham']]
In [251]: food_copy = food[:]
In [252]: food[1].pop()
Out[252]: 'ham'
In [253]: food
Out[253]: ['spam', ['eggs']]
In [256]: food.pop(0)
Out[256]: 'spam'
In [257]: food
Out[257]: [['eggs']]
In [258]: food_copy
Out[258]: ['spam', ['eggs']]
Copies can solve problems¶
Consider this common pattern:
for x in somelist:
if should_be_removed(x):
somelist.remove(x)
This looks benign enough, but changing a list while you are iterating over it can be the cause of some pernicious bugs.
The Problem¶
For example:
In [27]: l = list(range(10))
In [28]: l
Out[28]: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
In [29]: for item in l:
....: l.remove(item)
....:
In [30]: l
Out[30]: [1, 3, 5, 7, 9]
Was that what you expected?
The Solution¶
Iterate over a copy, and mutate the original:
In [33]: l = list(range(10))
In [34]: for item in l[:]:
....: l.remove(item)
....:
In [35]: l
Out[35]: []
Miscellaneous List Methods¶
These methods change a list in place and are not available on immutable sequence types.
.reverse()
In [129]: food = ['spam', 'eggs', 'ham']
In [130]: food.reverse()
In [131]: food
Out[131]: ['ham', 'eggs', 'spam']
.sort()
In [132]: food.sort()
In [133]: food
Out[133]: ['eggs', 'ham', 'spam']
Because these methods mutate the list in place, they have a return value of None
Custom Sorting¶
.sort()
can take an optional key
parameter.
It should be a function that takes one parameter (list items one at a time) and returns something that can be used for sorting:
In [137]: def third_letter(string):
.....: return string[2]
.....:
In [138]: food.sort(key=third_letter)
In [139]: food
Out[139]: ['spam', 'eggs', 'ham']
You end up with the list sorted by the third letter in each element.
List Performance¶
- indexing is fast and constant time: O(1)
x in l
is proportional to n: O(n)- visiting all is proportional to n: O(n)
- operating on the end of list is fast and constant time: O(1)
- append(), pop()
- operating on the front (or middle) of the list depends on n: O(n)
pop(0)
,insert(0, v)
- But, reversing is fast.
Also, collections.deque
What the heck does this O() thing mean? That is known as “big O” notation for time complexity. What it does is provide an indication of how much more time an operation will take depending on how many items the operation is acting on.
Check out the Python wiki entry on Time Complexity for more info:
Choosing Lists or Tuples¶
Here are a few guidelines on when to choose a list or a tuple:
- If it needs to be mutable: list
- If it needs to be immutable: tuple
- provides safety when passing to a function (and as a key in a dict)
Otherwise … taste and convention.
Convention¶
Lists are homogeneous collections: – they alway contain values of the same type – they simplify iterating, sorting, etc
Tuples are mixed types: – they group multiple values into one logical thing – they are similar to simple C structs.
Other Considerations¶
- Do you need to do the same operation to each element?
- list
- Is there a small collection of values which make a single logical item?
- tuple
- Do you want to document that these values won’t change?
- tuple
- Do you want to build it iteratively?
- list
- Do you need to transform, filter, etc?
- list
More Documentation¶
For more information, read the list docs:
https://docs.python.org/3.6/library/stdtypes.html#mutable-sequence-types
(actually any mutable sequence….)