Map Filter and Reduce
map
, filter
, and reduce
are considered key parts of “functional programming”
But there is no real consensus about what “functional” means.
But Python does support these “classic” functional methods.
map
map
“maps” a function onto a sequence of objects – It applies the function to each item in the sequence, returning an “iterable” “map object”.
The map object delays evaluation until you iterate over it. That way you can pass it to another map, or use it in a for loop, without creating an unnecessary copy of the list.
In [4]: l = [2, 5, 7, 12, 6, 4]
In [5]: def fun(x):
...: return x*2 + 10
...:
In [6]: map(fun, l)
Out[6]: <map at 0x1046f2ac8>
In [7]: list(map(fun, l))
Out[7]: [14, 20, 24, 34, 22, 18]
And if it’s a small function, and you only need it once, this is a great use for lambda
In [26]: list(map(lambda x: x*2 + 10, l))
Out[26]: [14, 20, 24, 34, 22, 18]
filter
filter
“filters” a sequence of objects with a boolean function –
It keeps only those for which the function is True – filtering out the rest.
It similarly returns an iterable object.
To get only the even numbers:
In [27]: l = [2, 5, 7, 12, 6, 4]
In [28]: list(filter(lambda x: not x%2, l))
Out[28]: [2, 12, 6, 4]
If you pass None
to filter()
, you get only items that evaluate to true:
In [1]: l = [1, 0, 2.3, 0.0, 'text', '', [1,2], [], False, True, None ]
In [2]: list(filter(None, l))
Out[2]: [1, 2.3, 'text', [1, 2], True]
reduce
reduce
“reduces” a sequence of objects to a single object with a function that combines two arguments.
In python3, reduce has been a bit hidden in the “functools” module:
from functools import reduce
To get the sum:
In [11]: from functools import reduce
In [12]: l = [2, 5, 7, 12, 6, 4]
In [13]: reduce(lambda x,y: x+y, l)
Out[13]: 36
To get the product:
In [32]: reduce(lambda x,y: x*y, l)
Out[32]: 20160
or
In [13]: import operator
In [14]: reduce(operator.mul, l)
Out[14]: 20160
Comprehensions
Couldn’t you do all this with comprehensions?
Yes:
In [33]: [x+2 + 10 for x in l]
Out[33]: [14, 17, 19, 24, 18, 16]
In [34]: [x for x in l if not x%2]
Out[34]: [2, 12, 6, 4]
In [6]: l
Out[6]: [1, 0, 2.3, 0.0, 'text', '', [1, 2], [], False, True, None]
In [7]: [i for i in l if i]
Out[7]: [1, 2.3, 'text', [1, 2], True]
(Except Reduce)
But Guido thinks almost all uses of reduce are really sum()
. In fact, that’s why it got moved out of built-ins into the functools
module.
Functional Programming
Comprehensions and map, filter, reduce are all “functional programming” approaches.
map, filter
and reduce
pre-date comprehensions in Python’s history
Some people like that syntax better.
And “map-reduce” is a big concept these days for parallel processing of “Big Data” in NoSQL databases.
(Hadoop, MongoDB, etc.)