Composing functions in python
I have an array of functions and I'm trying to produce one function which consists of the composition of the elements in my array.
My approach is:
def compose(list):
if len(list) == 1:
return lambda x:list[0](x)
list.reverse()
final=lambda x:x
for f in list:
final=lambda x:f(final(x))
return final
This method doesn't seems to be working, help will be appreciated.
(I'm reversing the list because this is the order of composition I want the functions to be)
python composition
add a comment |
I have an array of functions and I'm trying to produce one function which consists of the composition of the elements in my array.
My approach is:
def compose(list):
if len(list) == 1:
return lambda x:list[0](x)
list.reverse()
final=lambda x:x
for f in list:
final=lambda x:f(final(x))
return final
This method doesn't seems to be working, help will be appreciated.
(I'm reversing the list because this is the order of composition I want the functions to be)
python composition
add a comment |
I have an array of functions and I'm trying to produce one function which consists of the composition of the elements in my array.
My approach is:
def compose(list):
if len(list) == 1:
return lambda x:list[0](x)
list.reverse()
final=lambda x:x
for f in list:
final=lambda x:f(final(x))
return final
This method doesn't seems to be working, help will be appreciated.
(I'm reversing the list because this is the order of composition I want the functions to be)
python composition
I have an array of functions and I'm trying to produce one function which consists of the composition of the elements in my array.
My approach is:
def compose(list):
if len(list) == 1:
return lambda x:list[0](x)
list.reverse()
final=lambda x:x
for f in list:
final=lambda x:f(final(x))
return final
This method doesn't seems to be working, help will be appreciated.
(I'm reversing the list because this is the order of composition I want the functions to be)
python composition
python composition
asked May 24 '13 at 16:10
StarlessStarless
101114
101114
add a comment |
add a comment |
10 Answers
10
active
oldest
votes
It doesn't work because all the anonymous functions you create in the loop refer to the same loop variable and therefore share its final value.
As a quick fix, you can replace the assignment with:
final = lambda x, f=f, final=final: f(final(x))
Or, you can return the lambda from a function:
def wrap(accum, f):
return lambda x: f(accum(x))
...
final = wrap(final, f)
To understand what's going on, try this experiment:
>>> l = [lambda: n for n in xrange(10)]
>>> [f() for f in l]
[9, 9, 9, 9, 9, 9, 9, 9, 9, 9]
This result surprises many people, who expect the result to be [0, 1, 2, ...]
. However, all the lambdas point to the same n
variable, and all refer to its final value, which is 9. In your case, all the versions of final
which are supposed to nest end up referring to the same f
and, even worse, to the same final
.
The topic of lambdas and for loops in Python has been already covered on SO.
Thanks for the answer, it indeed worked for me. I used the second method. Can you explain what do you mean by "final closures refer to the same f cell", and also can you please explain the first method.
– Starless
May 24 '13 at 16:33
@Starless I've updated the answer with a longer explanation.
– user4815162342
May 24 '13 at 17:54
Here's an interesting alternative. Replacel
withl = [lambda x=n: x for n in range(10)]
This produces[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
as one might expect.
– RussAbbott
Jan 6 at 4:04
@RussAbbott That's the gist of the "quick fix" proposed near the beginning of the answer. In that pattern the convention is to name the keyword the same as the variable you are capturing, e.g.lambda n=n: ...
.
– user4815162342
Jan 6 at 7:53
add a comment |
The easiest approach would be first to write a composition of 2 functions:
def compose2(f, g):
return lambda *a, **kw: f(g(*a, **kw))
And then use reduce
to compose more functions:
def compose(*fs):
return reduce(compose2, fs)
Or you can use some library, which already contains compose function.
This is going to create a shadow function for every function infs
. I don’t know how much functions in Python are resource intensive, but that seems wasteful. Instead, see other solution by Imanol Luengo:def compose(*funcs): return lambda x: reduce(lambda acc, f: f(acc), funcs, x)
(stackoverflow.com/a/16739663/216138)
– Amir
Nov 14 '18 at 12:00
1
You can bench it, but your solution will be probably slower. For most common case of 2 functions mine is zero cost.
– Suor
Nov 15 '18 at 12:13
add a comment |
def compose (*functions):
def inner(arg):
for f in reversed(functions):
arg = f(arg)
return arg
return inner
Example:
>>> def square (x):
return x ** 2
>>> def increment (x):
return x + 1
>>> def half (x):
return x / 2
>>> composed = compose(square, increment, half) # square(increment(half(x)))
>>> composed(5) # square(increment(half(5))) = square(increment(2.5)) = square(3.5) = 12,25
12.25
Can you show how (/is it even possible) to add an aggregation step - presuming the chained functions are operating on collections?
– javadba
Oct 20 '18 at 21:54
@javadba I’m not sure what you mean. Can you give an example for what you would like to do?
– poke
Oct 21 '18 at 15:35
Consider the functions might be :(add 5 to x, mult by 3, *find top 3*, *sum*)
. the "top3" and "sum" are aggregations that I don't know how to insert into the composition.
– javadba
Oct 21 '18 at 15:38
@javadba You could surely do that, although I would say that it looks a bit complicated then:compose(sum, lambda x: sorted(x, reverse=True)[:3], lambda x: map(lambda y: y * 3, x), lambda x: map(lambda y: y + 5, x))
– You could also justmap
once with a composed function:compose(sum, lambda x: sorted(x, reverse=True)[:3], lambda x: map(compose(lambda y: y * 3, lambda y: y + 5), x))
. So if you named them nicely, it could look like this:compose(sum, top3, lambda x: map(compose(times3, plus5), x))
. You could also get rid of thatlambda
by usingfunctools.partial
.
– poke
Oct 21 '18 at 19:33
add a comment |
Recursive implementation
Here's a fairly elegant recursive implementation, which uses features of Python 3 for clarity:
def strict_compose(*funcs):
*funcs, penultimate, last = funcs
if funcs:
penultimate = strict_compose(*funcs, penultimate)
return lambda *args, **kwargs: penultimate(last(*args, **kwargs))
Python 2 compatible version:
def strict_compose2(*funcs):
if len(funcs) > 2:
penultimate = strict_compose2(*funcs[:-1])
else:
penultimate = funcs[-2]
return lambda *args, **kwargs: penultimate(funcs[-1](*args, **kwargs))
This is an earlier version which uses lazy evaluation of the recursion:
def lazy_recursive_compose(*funcs):
def inner(*args, _funcs=funcs, **kwargs):
if len(_funcs) > 1:
return inner(_funcs[-1](*args, **kwargs), _funcs=_funcs[:-1])
else:
return _funcs[0](*args, **kwargs)
return inner
Both would seem to make a new tuple and dict of arguments each recursive call.
Comparison of all suggestions:
Let's test some of these implementations and determine which is most performant, first some single argument functions (Thank you poke):
def square(x):
return x ** 2
def increment(x):
return x + 1
def half(x):
return x / 2
Here's our implementations, I suspect my iterative version is the second most efficient (manual compose will naturally be fastest), but that may be in part due to it sidestepping the difficulty of passing any number of arguments or keyword arguments between functions - in most cases we'll only see the trivial one argument being passed.
from functools import reduce
def strict_recursive_compose(*funcs):
*funcs, penultimate, last = funcs
if funcs:
penultimate = strict_recursive_compose(*funcs, penultimate)
return lambda *args, **kwargs: penultimate(last(*args, **kwargs))
def strict_recursive_compose2(*funcs):
if len(funcs) > 2:
penultimate = strict_recursive_compose2(*funcs[:-1])
else:
penultimate = funcs[-2]
return lambda *args, **kwargs: penultimate(funcs[-1](*args, **kwargs))
def lazy_recursive_compose(*funcs):
def inner(*args, _funcs=funcs, **kwargs):
if len(_funcs) > 1:
return inner(_funcs[-1](*args, **kwargs), _funcs=_funcs[:-1])
else:
return _funcs[0](*args, **kwargs)
return inner
def iterative_compose(*functions):
"""my implementation, only accepts one argument."""
def inner(arg):
for f in reversed(functions):
arg = f(arg)
return arg
return inner
def _compose2(f, g):
return lambda *a, **kw: f(g(*a, **kw))
def reduce_compose1(*fs):
return reduce(_compose2, fs)
def reduce_compose2(*funcs):
"""bug fixed - added reversed()"""
return lambda x: reduce(lambda acc, f: f(acc), reversed(funcs), x)
And to test these:
import timeit
def manual_compose(n):
return square(increment(half(n)))
composes = (strict_recursive_compose, strict_recursive_compose2,
lazy_recursive_compose, iterative_compose,
reduce_compose1, reduce_compose2)
print('manual compose', min(timeit.repeat(lambda: manual_compose(5))), manual_compose(5))
for compose in composes:
fn = compose(square, increment, half)
result = min(timeit.repeat(lambda: fn(5)))
print(compose.__name__, result, fn(5))
Results
And we get the following output (same magnitude and proportion in Python 2 and 3):
manual compose 0.4963762479601428 12.25
strict_recursive_compose 0.6564744340721518 12.25
strict_recursive_compose2 0.7216697579715401 12.25
lazy_recursive_compose 1.260614730999805 12.25
iterative_compose 0.614982972969301 12.25
reduce_compose1 0.6768529079854488 12.25
reduce_compose2 0.9890829260693863 12.25
And my expectations were confirmed: the fastest is of course, manual function composition followed by the iterative implementation. The lazy recursive version is much slower - likely since a new stack frame is created by each function call and a new tuple of functions is created for each function.
For a better and perhaps more realistic comparison, if you remove **kwargs
and change *args
to arg
in the functions, the ones that used them will be more performant, and we can better compare apples to apples - here, aside from manual composition, reduce_compose1 wins followed by the strict_recursive_compose:
manual compose 0.443808660027571 12.25
strict_recursive_compose 0.5409777010791004 12.25
strict_recursive_compose2 0.5698030130006373 12.25
lazy_recursive_compose 1.0381018499610946 12.25
iterative_compose 0.619289995986037 12.25
reduce_compose1 0.49532539502251893 12.25
reduce_compose2 0.9633988010464236 12.25
Functions with just one arg:
def strict_recursive_compose(*funcs):
*funcs, penultimate, last = funcs
if funcs:
penultimate = strict_recursive_compose(*funcs, penultimate)
return lambda arg: penultimate(last(arg))
def strict_recursive_compose2(*funcs):
if len(funcs) > 2:
penultimate = strict_recursive_compose2(*funcs[:-1])
else:
penultimate = funcs[-2]
return lambda arg: penultimate(funcs[-1](arg))
def lazy_recursive_compose(*funcs):
def inner(arg, _funcs=funcs):
if len(_funcs) > 1:
return inner(_funcs[-1](arg), _funcs=_funcs[:-1])
else:
return _funcs[0](arg)
return inner
def iterative_compose(*functions):
"""my implementation, only accepts one argument."""
def inner(arg):
for f in reversed(functions):
arg = f(arg)
return arg
return inner
def _compose2(f, g):
return lambda arg: f(g(arg))
def reduce_compose1(*fs):
return reduce(_compose2, fs)
def reduce_compose2(*funcs):
"""bug fixed - added reversed()"""
return lambda x: reduce(lambda acc, f: f(acc), reversed(funcs), x)
add a comment |
One liner:
compose = lambda *F: reduce(lambda f, g: lambda x: f(g(x)), F)
Example usage:
f1 = lambda x: x+3
f2 = lambda x: x*2
f3 = lambda x: x-1
g = compose(f1, f2, f3)
assert(g(7) == 15)
add a comment |
You can also create an array of functions and use reduce:
def f1(x): return x+1
def f2(x): return x+2
def f3(x): return x+3
x = 5
# Will print f3(f2(f1(x)))
print reduce(lambda acc, x: x(acc), [f1, f2, f3], x)
# As a function:
def compose(*funcs):
return lambda x: reduce(lambda acc, f: f(acc), funcs, x)
f = compose(f1, f2, f3)
Can you show how (/is it even possible) to add an aggregation step - presuming the chained functions are operating on collections?
– javadba
Oct 20 '18 at 21:56
add a comment |
Poke's answer is good, but you can also use the functional
package which comes with a compose method.
I was looking into implementing one myself
– Starless
May 24 '13 at 16:32
@Starless It has many recipes. Like any writer, you must read to improve what you write.
– Marcin
May 24 '13 at 16:33
FWIK, functional.compose only supports two arguments.
– georg
May 24 '13 at 17:31
@thg435 It has recipes for a multi-compose.
– Marcin
May 24 '13 at 17:57
@poke solution is convenient in that it takes the arguments ininfix
notation/order
– javadba
Oct 20 '18 at 21:55
add a comment |
The most reliable implementation I have found is in the 3rd party library toolz
. The compose
function from this library also deals with docstring for the composition of functions.
The source code is freely available. Below is a simple example of usage.
from toolz import compose
def f(x):
return x+1
def g(x):
return x*2
def h(x):
return x+3
res = compose(f, g, h)(5) # 17
add a comment |
pip install funcoperators
is another library to implement it that allows infix notation:
from funcoperators import compose
# display = lambda x: hex(ord(list(x)))
display = hex *compose* ord *compose* list
# also works as a function
display = compose(hex, ord, list)
pip install funcoperators https://pypi.org/project/funcoperators/
Disclaimer: I'm the creator of the module
add a comment |
This is my version
def compose(*fargs):
def inner(arg):
if not arg:
raise ValueError("Invalid argument")
if not all([callable(f) for f in fargs]):
raise TypeError("Function is not callable")
return reduce(lambda arg, func: func(arg), fargs, arg)
return inner
An example of how it's used
def calcMean(iterable):
return sum(iterable) / len(iterable)
def formatMean(mean):
return round(float(mean), 2)
def adder(val, value):
return val + value
def isEven(val):
return val % 2 == 0
if __name__ == '__main__':
# Ex1
rand_range = [random.randint(0, 10000) for x in range(0, 10000)]
isRandIntEven = compose(calcMean, formatMean,
partial(adder, value=0), math.floor.__call__, isEven)
print(isRandIntEven(rand_range))
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f16739290%2fcomposing-functions-in-python%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
10 Answers
10
active
oldest
votes
10 Answers
10
active
oldest
votes
active
oldest
votes
active
oldest
votes
It doesn't work because all the anonymous functions you create in the loop refer to the same loop variable and therefore share its final value.
As a quick fix, you can replace the assignment with:
final = lambda x, f=f, final=final: f(final(x))
Or, you can return the lambda from a function:
def wrap(accum, f):
return lambda x: f(accum(x))
...
final = wrap(final, f)
To understand what's going on, try this experiment:
>>> l = [lambda: n for n in xrange(10)]
>>> [f() for f in l]
[9, 9, 9, 9, 9, 9, 9, 9, 9, 9]
This result surprises many people, who expect the result to be [0, 1, 2, ...]
. However, all the lambdas point to the same n
variable, and all refer to its final value, which is 9. In your case, all the versions of final
which are supposed to nest end up referring to the same f
and, even worse, to the same final
.
The topic of lambdas and for loops in Python has been already covered on SO.
Thanks for the answer, it indeed worked for me. I used the second method. Can you explain what do you mean by "final closures refer to the same f cell", and also can you please explain the first method.
– Starless
May 24 '13 at 16:33
@Starless I've updated the answer with a longer explanation.
– user4815162342
May 24 '13 at 17:54
Here's an interesting alternative. Replacel
withl = [lambda x=n: x for n in range(10)]
This produces[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
as one might expect.
– RussAbbott
Jan 6 at 4:04
@RussAbbott That's the gist of the "quick fix" proposed near the beginning of the answer. In that pattern the convention is to name the keyword the same as the variable you are capturing, e.g.lambda n=n: ...
.
– user4815162342
Jan 6 at 7:53
add a comment |
It doesn't work because all the anonymous functions you create in the loop refer to the same loop variable and therefore share its final value.
As a quick fix, you can replace the assignment with:
final = lambda x, f=f, final=final: f(final(x))
Or, you can return the lambda from a function:
def wrap(accum, f):
return lambda x: f(accum(x))
...
final = wrap(final, f)
To understand what's going on, try this experiment:
>>> l = [lambda: n for n in xrange(10)]
>>> [f() for f in l]
[9, 9, 9, 9, 9, 9, 9, 9, 9, 9]
This result surprises many people, who expect the result to be [0, 1, 2, ...]
. However, all the lambdas point to the same n
variable, and all refer to its final value, which is 9. In your case, all the versions of final
which are supposed to nest end up referring to the same f
and, even worse, to the same final
.
The topic of lambdas and for loops in Python has been already covered on SO.
Thanks for the answer, it indeed worked for me. I used the second method. Can you explain what do you mean by "final closures refer to the same f cell", and also can you please explain the first method.
– Starless
May 24 '13 at 16:33
@Starless I've updated the answer with a longer explanation.
– user4815162342
May 24 '13 at 17:54
Here's an interesting alternative. Replacel
withl = [lambda x=n: x for n in range(10)]
This produces[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
as one might expect.
– RussAbbott
Jan 6 at 4:04
@RussAbbott That's the gist of the "quick fix" proposed near the beginning of the answer. In that pattern the convention is to name the keyword the same as the variable you are capturing, e.g.lambda n=n: ...
.
– user4815162342
Jan 6 at 7:53
add a comment |
It doesn't work because all the anonymous functions you create in the loop refer to the same loop variable and therefore share its final value.
As a quick fix, you can replace the assignment with:
final = lambda x, f=f, final=final: f(final(x))
Or, you can return the lambda from a function:
def wrap(accum, f):
return lambda x: f(accum(x))
...
final = wrap(final, f)
To understand what's going on, try this experiment:
>>> l = [lambda: n for n in xrange(10)]
>>> [f() for f in l]
[9, 9, 9, 9, 9, 9, 9, 9, 9, 9]
This result surprises many people, who expect the result to be [0, 1, 2, ...]
. However, all the lambdas point to the same n
variable, and all refer to its final value, which is 9. In your case, all the versions of final
which are supposed to nest end up referring to the same f
and, even worse, to the same final
.
The topic of lambdas and for loops in Python has been already covered on SO.
It doesn't work because all the anonymous functions you create in the loop refer to the same loop variable and therefore share its final value.
As a quick fix, you can replace the assignment with:
final = lambda x, f=f, final=final: f(final(x))
Or, you can return the lambda from a function:
def wrap(accum, f):
return lambda x: f(accum(x))
...
final = wrap(final, f)
To understand what's going on, try this experiment:
>>> l = [lambda: n for n in xrange(10)]
>>> [f() for f in l]
[9, 9, 9, 9, 9, 9, 9, 9, 9, 9]
This result surprises many people, who expect the result to be [0, 1, 2, ...]
. However, all the lambdas point to the same n
variable, and all refer to its final value, which is 9. In your case, all the versions of final
which are supposed to nest end up referring to the same f
and, even worse, to the same final
.
The topic of lambdas and for loops in Python has been already covered on SO.
edited Dec 20 '17 at 19:39
answered May 24 '13 at 16:16
user4815162342user4815162342
64k594151
64k594151
Thanks for the answer, it indeed worked for me. I used the second method. Can you explain what do you mean by "final closures refer to the same f cell", and also can you please explain the first method.
– Starless
May 24 '13 at 16:33
@Starless I've updated the answer with a longer explanation.
– user4815162342
May 24 '13 at 17:54
Here's an interesting alternative. Replacel
withl = [lambda x=n: x for n in range(10)]
This produces[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
as one might expect.
– RussAbbott
Jan 6 at 4:04
@RussAbbott That's the gist of the "quick fix" proposed near the beginning of the answer. In that pattern the convention is to name the keyword the same as the variable you are capturing, e.g.lambda n=n: ...
.
– user4815162342
Jan 6 at 7:53
add a comment |
Thanks for the answer, it indeed worked for me. I used the second method. Can you explain what do you mean by "final closures refer to the same f cell", and also can you please explain the first method.
– Starless
May 24 '13 at 16:33
@Starless I've updated the answer with a longer explanation.
– user4815162342
May 24 '13 at 17:54
Here's an interesting alternative. Replacel
withl = [lambda x=n: x for n in range(10)]
This produces[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
as one might expect.
– RussAbbott
Jan 6 at 4:04
@RussAbbott That's the gist of the "quick fix" proposed near the beginning of the answer. In that pattern the convention is to name the keyword the same as the variable you are capturing, e.g.lambda n=n: ...
.
– user4815162342
Jan 6 at 7:53
Thanks for the answer, it indeed worked for me. I used the second method. Can you explain what do you mean by "final closures refer to the same f cell", and also can you please explain the first method.
– Starless
May 24 '13 at 16:33
Thanks for the answer, it indeed worked for me. I used the second method. Can you explain what do you mean by "final closures refer to the same f cell", and also can you please explain the first method.
– Starless
May 24 '13 at 16:33
@Starless I've updated the answer with a longer explanation.
– user4815162342
May 24 '13 at 17:54
@Starless I've updated the answer with a longer explanation.
– user4815162342
May 24 '13 at 17:54
Here's an interesting alternative. Replace
l
with l = [lambda x=n: x for n in range(10)]
This produces [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
as one might expect.– RussAbbott
Jan 6 at 4:04
Here's an interesting alternative. Replace
l
with l = [lambda x=n: x for n in range(10)]
This produces [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
as one might expect.– RussAbbott
Jan 6 at 4:04
@RussAbbott That's the gist of the "quick fix" proposed near the beginning of the answer. In that pattern the convention is to name the keyword the same as the variable you are capturing, e.g.
lambda n=n: ...
.– user4815162342
Jan 6 at 7:53
@RussAbbott That's the gist of the "quick fix" proposed near the beginning of the answer. In that pattern the convention is to name the keyword the same as the variable you are capturing, e.g.
lambda n=n: ...
.– user4815162342
Jan 6 at 7:53
add a comment |
The easiest approach would be first to write a composition of 2 functions:
def compose2(f, g):
return lambda *a, **kw: f(g(*a, **kw))
And then use reduce
to compose more functions:
def compose(*fs):
return reduce(compose2, fs)
Or you can use some library, which already contains compose function.
This is going to create a shadow function for every function infs
. I don’t know how much functions in Python are resource intensive, but that seems wasteful. Instead, see other solution by Imanol Luengo:def compose(*funcs): return lambda x: reduce(lambda acc, f: f(acc), funcs, x)
(stackoverflow.com/a/16739663/216138)
– Amir
Nov 14 '18 at 12:00
1
You can bench it, but your solution will be probably slower. For most common case of 2 functions mine is zero cost.
– Suor
Nov 15 '18 at 12:13
add a comment |
The easiest approach would be first to write a composition of 2 functions:
def compose2(f, g):
return lambda *a, **kw: f(g(*a, **kw))
And then use reduce
to compose more functions:
def compose(*fs):
return reduce(compose2, fs)
Or you can use some library, which already contains compose function.
This is going to create a shadow function for every function infs
. I don’t know how much functions in Python are resource intensive, but that seems wasteful. Instead, see other solution by Imanol Luengo:def compose(*funcs): return lambda x: reduce(lambda acc, f: f(acc), funcs, x)
(stackoverflow.com/a/16739663/216138)
– Amir
Nov 14 '18 at 12:00
1
You can bench it, but your solution will be probably slower. For most common case of 2 functions mine is zero cost.
– Suor
Nov 15 '18 at 12:13
add a comment |
The easiest approach would be first to write a composition of 2 functions:
def compose2(f, g):
return lambda *a, **kw: f(g(*a, **kw))
And then use reduce
to compose more functions:
def compose(*fs):
return reduce(compose2, fs)
Or you can use some library, which already contains compose function.
The easiest approach would be first to write a composition of 2 functions:
def compose2(f, g):
return lambda *a, **kw: f(g(*a, **kw))
And then use reduce
to compose more functions:
def compose(*fs):
return reduce(compose2, fs)
Or you can use some library, which already contains compose function.
answered Jun 4 '14 at 20:43
SuorSuor
1,8321121
1,8321121
This is going to create a shadow function for every function infs
. I don’t know how much functions in Python are resource intensive, but that seems wasteful. Instead, see other solution by Imanol Luengo:def compose(*funcs): return lambda x: reduce(lambda acc, f: f(acc), funcs, x)
(stackoverflow.com/a/16739663/216138)
– Amir
Nov 14 '18 at 12:00
1
You can bench it, but your solution will be probably slower. For most common case of 2 functions mine is zero cost.
– Suor
Nov 15 '18 at 12:13
add a comment |
This is going to create a shadow function for every function infs
. I don’t know how much functions in Python are resource intensive, but that seems wasteful. Instead, see other solution by Imanol Luengo:def compose(*funcs): return lambda x: reduce(lambda acc, f: f(acc), funcs, x)
(stackoverflow.com/a/16739663/216138)
– Amir
Nov 14 '18 at 12:00
1
You can bench it, but your solution will be probably slower. For most common case of 2 functions mine is zero cost.
– Suor
Nov 15 '18 at 12:13
This is going to create a shadow function for every function in
fs
. I don’t know how much functions in Python are resource intensive, but that seems wasteful. Instead, see other solution by Imanol Luengo: def compose(*funcs): return lambda x: reduce(lambda acc, f: f(acc), funcs, x)
(stackoverflow.com/a/16739663/216138)– Amir
Nov 14 '18 at 12:00
This is going to create a shadow function for every function in
fs
. I don’t know how much functions in Python are resource intensive, but that seems wasteful. Instead, see other solution by Imanol Luengo: def compose(*funcs): return lambda x: reduce(lambda acc, f: f(acc), funcs, x)
(stackoverflow.com/a/16739663/216138)– Amir
Nov 14 '18 at 12:00
1
1
You can bench it, but your solution will be probably slower. For most common case of 2 functions mine is zero cost.
– Suor
Nov 15 '18 at 12:13
You can bench it, but your solution will be probably slower. For most common case of 2 functions mine is zero cost.
– Suor
Nov 15 '18 at 12:13
add a comment |
def compose (*functions):
def inner(arg):
for f in reversed(functions):
arg = f(arg)
return arg
return inner
Example:
>>> def square (x):
return x ** 2
>>> def increment (x):
return x + 1
>>> def half (x):
return x / 2
>>> composed = compose(square, increment, half) # square(increment(half(x)))
>>> composed(5) # square(increment(half(5))) = square(increment(2.5)) = square(3.5) = 12,25
12.25
Can you show how (/is it even possible) to add an aggregation step - presuming the chained functions are operating on collections?
– javadba
Oct 20 '18 at 21:54
@javadba I’m not sure what you mean. Can you give an example for what you would like to do?
– poke
Oct 21 '18 at 15:35
Consider the functions might be :(add 5 to x, mult by 3, *find top 3*, *sum*)
. the "top3" and "sum" are aggregations that I don't know how to insert into the composition.
– javadba
Oct 21 '18 at 15:38
@javadba You could surely do that, although I would say that it looks a bit complicated then:compose(sum, lambda x: sorted(x, reverse=True)[:3], lambda x: map(lambda y: y * 3, x), lambda x: map(lambda y: y + 5, x))
– You could also justmap
once with a composed function:compose(sum, lambda x: sorted(x, reverse=True)[:3], lambda x: map(compose(lambda y: y * 3, lambda y: y + 5), x))
. So if you named them nicely, it could look like this:compose(sum, top3, lambda x: map(compose(times3, plus5), x))
. You could also get rid of thatlambda
by usingfunctools.partial
.
– poke
Oct 21 '18 at 19:33
add a comment |
def compose (*functions):
def inner(arg):
for f in reversed(functions):
arg = f(arg)
return arg
return inner
Example:
>>> def square (x):
return x ** 2
>>> def increment (x):
return x + 1
>>> def half (x):
return x / 2
>>> composed = compose(square, increment, half) # square(increment(half(x)))
>>> composed(5) # square(increment(half(5))) = square(increment(2.5)) = square(3.5) = 12,25
12.25
Can you show how (/is it even possible) to add an aggregation step - presuming the chained functions are operating on collections?
– javadba
Oct 20 '18 at 21:54
@javadba I’m not sure what you mean. Can you give an example for what you would like to do?
– poke
Oct 21 '18 at 15:35
Consider the functions might be :(add 5 to x, mult by 3, *find top 3*, *sum*)
. the "top3" and "sum" are aggregations that I don't know how to insert into the composition.
– javadba
Oct 21 '18 at 15:38
@javadba You could surely do that, although I would say that it looks a bit complicated then:compose(sum, lambda x: sorted(x, reverse=True)[:3], lambda x: map(lambda y: y * 3, x), lambda x: map(lambda y: y + 5, x))
– You could also justmap
once with a composed function:compose(sum, lambda x: sorted(x, reverse=True)[:3], lambda x: map(compose(lambda y: y * 3, lambda y: y + 5), x))
. So if you named them nicely, it could look like this:compose(sum, top3, lambda x: map(compose(times3, plus5), x))
. You could also get rid of thatlambda
by usingfunctools.partial
.
– poke
Oct 21 '18 at 19:33
add a comment |
def compose (*functions):
def inner(arg):
for f in reversed(functions):
arg = f(arg)
return arg
return inner
Example:
>>> def square (x):
return x ** 2
>>> def increment (x):
return x + 1
>>> def half (x):
return x / 2
>>> composed = compose(square, increment, half) # square(increment(half(x)))
>>> composed(5) # square(increment(half(5))) = square(increment(2.5)) = square(3.5) = 12,25
12.25
def compose (*functions):
def inner(arg):
for f in reversed(functions):
arg = f(arg)
return arg
return inner
Example:
>>> def square (x):
return x ** 2
>>> def increment (x):
return x + 1
>>> def half (x):
return x / 2
>>> composed = compose(square, increment, half) # square(increment(half(x)))
>>> composed(5) # square(increment(half(5))) = square(increment(2.5)) = square(3.5) = 12,25
12.25
answered May 24 '13 at 16:17
pokepoke
216k46336399
216k46336399
Can you show how (/is it even possible) to add an aggregation step - presuming the chained functions are operating on collections?
– javadba
Oct 20 '18 at 21:54
@javadba I’m not sure what you mean. Can you give an example for what you would like to do?
– poke
Oct 21 '18 at 15:35
Consider the functions might be :(add 5 to x, mult by 3, *find top 3*, *sum*)
. the "top3" and "sum" are aggregations that I don't know how to insert into the composition.
– javadba
Oct 21 '18 at 15:38
@javadba You could surely do that, although I would say that it looks a bit complicated then:compose(sum, lambda x: sorted(x, reverse=True)[:3], lambda x: map(lambda y: y * 3, x), lambda x: map(lambda y: y + 5, x))
– You could also justmap
once with a composed function:compose(sum, lambda x: sorted(x, reverse=True)[:3], lambda x: map(compose(lambda y: y * 3, lambda y: y + 5), x))
. So if you named them nicely, it could look like this:compose(sum, top3, lambda x: map(compose(times3, plus5), x))
. You could also get rid of thatlambda
by usingfunctools.partial
.
– poke
Oct 21 '18 at 19:33
add a comment |
Can you show how (/is it even possible) to add an aggregation step - presuming the chained functions are operating on collections?
– javadba
Oct 20 '18 at 21:54
@javadba I’m not sure what you mean. Can you give an example for what you would like to do?
– poke
Oct 21 '18 at 15:35
Consider the functions might be :(add 5 to x, mult by 3, *find top 3*, *sum*)
. the "top3" and "sum" are aggregations that I don't know how to insert into the composition.
– javadba
Oct 21 '18 at 15:38
@javadba You could surely do that, although I would say that it looks a bit complicated then:compose(sum, lambda x: sorted(x, reverse=True)[:3], lambda x: map(lambda y: y * 3, x), lambda x: map(lambda y: y + 5, x))
– You could also justmap
once with a composed function:compose(sum, lambda x: sorted(x, reverse=True)[:3], lambda x: map(compose(lambda y: y * 3, lambda y: y + 5), x))
. So if you named them nicely, it could look like this:compose(sum, top3, lambda x: map(compose(times3, plus5), x))
. You could also get rid of thatlambda
by usingfunctools.partial
.
– poke
Oct 21 '18 at 19:33
Can you show how (/is it even possible) to add an aggregation step - presuming the chained functions are operating on collections?
– javadba
Oct 20 '18 at 21:54
Can you show how (/is it even possible) to add an aggregation step - presuming the chained functions are operating on collections?
– javadba
Oct 20 '18 at 21:54
@javadba I’m not sure what you mean. Can you give an example for what you would like to do?
– poke
Oct 21 '18 at 15:35
@javadba I’m not sure what you mean. Can you give an example for what you would like to do?
– poke
Oct 21 '18 at 15:35
Consider the functions might be :
(add 5 to x, mult by 3, *find top 3*, *sum*)
. the "top3" and "sum" are aggregations that I don't know how to insert into the composition.– javadba
Oct 21 '18 at 15:38
Consider the functions might be :
(add 5 to x, mult by 3, *find top 3*, *sum*)
. the "top3" and "sum" are aggregations that I don't know how to insert into the composition.– javadba
Oct 21 '18 at 15:38
@javadba You could surely do that, although I would say that it looks a bit complicated then:
compose(sum, lambda x: sorted(x, reverse=True)[:3], lambda x: map(lambda y: y * 3, x), lambda x: map(lambda y: y + 5, x))
– You could also just map
once with a composed function: compose(sum, lambda x: sorted(x, reverse=True)[:3], lambda x: map(compose(lambda y: y * 3, lambda y: y + 5), x))
. So if you named them nicely, it could look like this: compose(sum, top3, lambda x: map(compose(times3, plus5), x))
. You could also get rid of that lambda
by using functools.partial
.– poke
Oct 21 '18 at 19:33
@javadba You could surely do that, although I would say that it looks a bit complicated then:
compose(sum, lambda x: sorted(x, reverse=True)[:3], lambda x: map(lambda y: y * 3, x), lambda x: map(lambda y: y + 5, x))
– You could also just map
once with a composed function: compose(sum, lambda x: sorted(x, reverse=True)[:3], lambda x: map(compose(lambda y: y * 3, lambda y: y + 5), x))
. So if you named them nicely, it could look like this: compose(sum, top3, lambda x: map(compose(times3, plus5), x))
. You could also get rid of that lambda
by using functools.partial
.– poke
Oct 21 '18 at 19:33
add a comment |
Recursive implementation
Here's a fairly elegant recursive implementation, which uses features of Python 3 for clarity:
def strict_compose(*funcs):
*funcs, penultimate, last = funcs
if funcs:
penultimate = strict_compose(*funcs, penultimate)
return lambda *args, **kwargs: penultimate(last(*args, **kwargs))
Python 2 compatible version:
def strict_compose2(*funcs):
if len(funcs) > 2:
penultimate = strict_compose2(*funcs[:-1])
else:
penultimate = funcs[-2]
return lambda *args, **kwargs: penultimate(funcs[-1](*args, **kwargs))
This is an earlier version which uses lazy evaluation of the recursion:
def lazy_recursive_compose(*funcs):
def inner(*args, _funcs=funcs, **kwargs):
if len(_funcs) > 1:
return inner(_funcs[-1](*args, **kwargs), _funcs=_funcs[:-1])
else:
return _funcs[0](*args, **kwargs)
return inner
Both would seem to make a new tuple and dict of arguments each recursive call.
Comparison of all suggestions:
Let's test some of these implementations and determine which is most performant, first some single argument functions (Thank you poke):
def square(x):
return x ** 2
def increment(x):
return x + 1
def half(x):
return x / 2
Here's our implementations, I suspect my iterative version is the second most efficient (manual compose will naturally be fastest), but that may be in part due to it sidestepping the difficulty of passing any number of arguments or keyword arguments between functions - in most cases we'll only see the trivial one argument being passed.
from functools import reduce
def strict_recursive_compose(*funcs):
*funcs, penultimate, last = funcs
if funcs:
penultimate = strict_recursive_compose(*funcs, penultimate)
return lambda *args, **kwargs: penultimate(last(*args, **kwargs))
def strict_recursive_compose2(*funcs):
if len(funcs) > 2:
penultimate = strict_recursive_compose2(*funcs[:-1])
else:
penultimate = funcs[-2]
return lambda *args, **kwargs: penultimate(funcs[-1](*args, **kwargs))
def lazy_recursive_compose(*funcs):
def inner(*args, _funcs=funcs, **kwargs):
if len(_funcs) > 1:
return inner(_funcs[-1](*args, **kwargs), _funcs=_funcs[:-1])
else:
return _funcs[0](*args, **kwargs)
return inner
def iterative_compose(*functions):
"""my implementation, only accepts one argument."""
def inner(arg):
for f in reversed(functions):
arg = f(arg)
return arg
return inner
def _compose2(f, g):
return lambda *a, **kw: f(g(*a, **kw))
def reduce_compose1(*fs):
return reduce(_compose2, fs)
def reduce_compose2(*funcs):
"""bug fixed - added reversed()"""
return lambda x: reduce(lambda acc, f: f(acc), reversed(funcs), x)
And to test these:
import timeit
def manual_compose(n):
return square(increment(half(n)))
composes = (strict_recursive_compose, strict_recursive_compose2,
lazy_recursive_compose, iterative_compose,
reduce_compose1, reduce_compose2)
print('manual compose', min(timeit.repeat(lambda: manual_compose(5))), manual_compose(5))
for compose in composes:
fn = compose(square, increment, half)
result = min(timeit.repeat(lambda: fn(5)))
print(compose.__name__, result, fn(5))
Results
And we get the following output (same magnitude and proportion in Python 2 and 3):
manual compose 0.4963762479601428 12.25
strict_recursive_compose 0.6564744340721518 12.25
strict_recursive_compose2 0.7216697579715401 12.25
lazy_recursive_compose 1.260614730999805 12.25
iterative_compose 0.614982972969301 12.25
reduce_compose1 0.6768529079854488 12.25
reduce_compose2 0.9890829260693863 12.25
And my expectations were confirmed: the fastest is of course, manual function composition followed by the iterative implementation. The lazy recursive version is much slower - likely since a new stack frame is created by each function call and a new tuple of functions is created for each function.
For a better and perhaps more realistic comparison, if you remove **kwargs
and change *args
to arg
in the functions, the ones that used them will be more performant, and we can better compare apples to apples - here, aside from manual composition, reduce_compose1 wins followed by the strict_recursive_compose:
manual compose 0.443808660027571 12.25
strict_recursive_compose 0.5409777010791004 12.25
strict_recursive_compose2 0.5698030130006373 12.25
lazy_recursive_compose 1.0381018499610946 12.25
iterative_compose 0.619289995986037 12.25
reduce_compose1 0.49532539502251893 12.25
reduce_compose2 0.9633988010464236 12.25
Functions with just one arg:
def strict_recursive_compose(*funcs):
*funcs, penultimate, last = funcs
if funcs:
penultimate = strict_recursive_compose(*funcs, penultimate)
return lambda arg: penultimate(last(arg))
def strict_recursive_compose2(*funcs):
if len(funcs) > 2:
penultimate = strict_recursive_compose2(*funcs[:-1])
else:
penultimate = funcs[-2]
return lambda arg: penultimate(funcs[-1](arg))
def lazy_recursive_compose(*funcs):
def inner(arg, _funcs=funcs):
if len(_funcs) > 1:
return inner(_funcs[-1](arg), _funcs=_funcs[:-1])
else:
return _funcs[0](arg)
return inner
def iterative_compose(*functions):
"""my implementation, only accepts one argument."""
def inner(arg):
for f in reversed(functions):
arg = f(arg)
return arg
return inner
def _compose2(f, g):
return lambda arg: f(g(arg))
def reduce_compose1(*fs):
return reduce(_compose2, fs)
def reduce_compose2(*funcs):
"""bug fixed - added reversed()"""
return lambda x: reduce(lambda acc, f: f(acc), reversed(funcs), x)
add a comment |
Recursive implementation
Here's a fairly elegant recursive implementation, which uses features of Python 3 for clarity:
def strict_compose(*funcs):
*funcs, penultimate, last = funcs
if funcs:
penultimate = strict_compose(*funcs, penultimate)
return lambda *args, **kwargs: penultimate(last(*args, **kwargs))
Python 2 compatible version:
def strict_compose2(*funcs):
if len(funcs) > 2:
penultimate = strict_compose2(*funcs[:-1])
else:
penultimate = funcs[-2]
return lambda *args, **kwargs: penultimate(funcs[-1](*args, **kwargs))
This is an earlier version which uses lazy evaluation of the recursion:
def lazy_recursive_compose(*funcs):
def inner(*args, _funcs=funcs, **kwargs):
if len(_funcs) > 1:
return inner(_funcs[-1](*args, **kwargs), _funcs=_funcs[:-1])
else:
return _funcs[0](*args, **kwargs)
return inner
Both would seem to make a new tuple and dict of arguments each recursive call.
Comparison of all suggestions:
Let's test some of these implementations and determine which is most performant, first some single argument functions (Thank you poke):
def square(x):
return x ** 2
def increment(x):
return x + 1
def half(x):
return x / 2
Here's our implementations, I suspect my iterative version is the second most efficient (manual compose will naturally be fastest), but that may be in part due to it sidestepping the difficulty of passing any number of arguments or keyword arguments between functions - in most cases we'll only see the trivial one argument being passed.
from functools import reduce
def strict_recursive_compose(*funcs):
*funcs, penultimate, last = funcs
if funcs:
penultimate = strict_recursive_compose(*funcs, penultimate)
return lambda *args, **kwargs: penultimate(last(*args, **kwargs))
def strict_recursive_compose2(*funcs):
if len(funcs) > 2:
penultimate = strict_recursive_compose2(*funcs[:-1])
else:
penultimate = funcs[-2]
return lambda *args, **kwargs: penultimate(funcs[-1](*args, **kwargs))
def lazy_recursive_compose(*funcs):
def inner(*args, _funcs=funcs, **kwargs):
if len(_funcs) > 1:
return inner(_funcs[-1](*args, **kwargs), _funcs=_funcs[:-1])
else:
return _funcs[0](*args, **kwargs)
return inner
def iterative_compose(*functions):
"""my implementation, only accepts one argument."""
def inner(arg):
for f in reversed(functions):
arg = f(arg)
return arg
return inner
def _compose2(f, g):
return lambda *a, **kw: f(g(*a, **kw))
def reduce_compose1(*fs):
return reduce(_compose2, fs)
def reduce_compose2(*funcs):
"""bug fixed - added reversed()"""
return lambda x: reduce(lambda acc, f: f(acc), reversed(funcs), x)
And to test these:
import timeit
def manual_compose(n):
return square(increment(half(n)))
composes = (strict_recursive_compose, strict_recursive_compose2,
lazy_recursive_compose, iterative_compose,
reduce_compose1, reduce_compose2)
print('manual compose', min(timeit.repeat(lambda: manual_compose(5))), manual_compose(5))
for compose in composes:
fn = compose(square, increment, half)
result = min(timeit.repeat(lambda: fn(5)))
print(compose.__name__, result, fn(5))
Results
And we get the following output (same magnitude and proportion in Python 2 and 3):
manual compose 0.4963762479601428 12.25
strict_recursive_compose 0.6564744340721518 12.25
strict_recursive_compose2 0.7216697579715401 12.25
lazy_recursive_compose 1.260614730999805 12.25
iterative_compose 0.614982972969301 12.25
reduce_compose1 0.6768529079854488 12.25
reduce_compose2 0.9890829260693863 12.25
And my expectations were confirmed: the fastest is of course, manual function composition followed by the iterative implementation. The lazy recursive version is much slower - likely since a new stack frame is created by each function call and a new tuple of functions is created for each function.
For a better and perhaps more realistic comparison, if you remove **kwargs
and change *args
to arg
in the functions, the ones that used them will be more performant, and we can better compare apples to apples - here, aside from manual composition, reduce_compose1 wins followed by the strict_recursive_compose:
manual compose 0.443808660027571 12.25
strict_recursive_compose 0.5409777010791004 12.25
strict_recursive_compose2 0.5698030130006373 12.25
lazy_recursive_compose 1.0381018499610946 12.25
iterative_compose 0.619289995986037 12.25
reduce_compose1 0.49532539502251893 12.25
reduce_compose2 0.9633988010464236 12.25
Functions with just one arg:
def strict_recursive_compose(*funcs):
*funcs, penultimate, last = funcs
if funcs:
penultimate = strict_recursive_compose(*funcs, penultimate)
return lambda arg: penultimate(last(arg))
def strict_recursive_compose2(*funcs):
if len(funcs) > 2:
penultimate = strict_recursive_compose2(*funcs[:-1])
else:
penultimate = funcs[-2]
return lambda arg: penultimate(funcs[-1](arg))
def lazy_recursive_compose(*funcs):
def inner(arg, _funcs=funcs):
if len(_funcs) > 1:
return inner(_funcs[-1](arg), _funcs=_funcs[:-1])
else:
return _funcs[0](arg)
return inner
def iterative_compose(*functions):
"""my implementation, only accepts one argument."""
def inner(arg):
for f in reversed(functions):
arg = f(arg)
return arg
return inner
def _compose2(f, g):
return lambda arg: f(g(arg))
def reduce_compose1(*fs):
return reduce(_compose2, fs)
def reduce_compose2(*funcs):
"""bug fixed - added reversed()"""
return lambda x: reduce(lambda acc, f: f(acc), reversed(funcs), x)
add a comment |
Recursive implementation
Here's a fairly elegant recursive implementation, which uses features of Python 3 for clarity:
def strict_compose(*funcs):
*funcs, penultimate, last = funcs
if funcs:
penultimate = strict_compose(*funcs, penultimate)
return lambda *args, **kwargs: penultimate(last(*args, **kwargs))
Python 2 compatible version:
def strict_compose2(*funcs):
if len(funcs) > 2:
penultimate = strict_compose2(*funcs[:-1])
else:
penultimate = funcs[-2]
return lambda *args, **kwargs: penultimate(funcs[-1](*args, **kwargs))
This is an earlier version which uses lazy evaluation of the recursion:
def lazy_recursive_compose(*funcs):
def inner(*args, _funcs=funcs, **kwargs):
if len(_funcs) > 1:
return inner(_funcs[-1](*args, **kwargs), _funcs=_funcs[:-1])
else:
return _funcs[0](*args, **kwargs)
return inner
Both would seem to make a new tuple and dict of arguments each recursive call.
Comparison of all suggestions:
Let's test some of these implementations and determine which is most performant, first some single argument functions (Thank you poke):
def square(x):
return x ** 2
def increment(x):
return x + 1
def half(x):
return x / 2
Here's our implementations, I suspect my iterative version is the second most efficient (manual compose will naturally be fastest), but that may be in part due to it sidestepping the difficulty of passing any number of arguments or keyword arguments between functions - in most cases we'll only see the trivial one argument being passed.
from functools import reduce
def strict_recursive_compose(*funcs):
*funcs, penultimate, last = funcs
if funcs:
penultimate = strict_recursive_compose(*funcs, penultimate)
return lambda *args, **kwargs: penultimate(last(*args, **kwargs))
def strict_recursive_compose2(*funcs):
if len(funcs) > 2:
penultimate = strict_recursive_compose2(*funcs[:-1])
else:
penultimate = funcs[-2]
return lambda *args, **kwargs: penultimate(funcs[-1](*args, **kwargs))
def lazy_recursive_compose(*funcs):
def inner(*args, _funcs=funcs, **kwargs):
if len(_funcs) > 1:
return inner(_funcs[-1](*args, **kwargs), _funcs=_funcs[:-1])
else:
return _funcs[0](*args, **kwargs)
return inner
def iterative_compose(*functions):
"""my implementation, only accepts one argument."""
def inner(arg):
for f in reversed(functions):
arg = f(arg)
return arg
return inner
def _compose2(f, g):
return lambda *a, **kw: f(g(*a, **kw))
def reduce_compose1(*fs):
return reduce(_compose2, fs)
def reduce_compose2(*funcs):
"""bug fixed - added reversed()"""
return lambda x: reduce(lambda acc, f: f(acc), reversed(funcs), x)
And to test these:
import timeit
def manual_compose(n):
return square(increment(half(n)))
composes = (strict_recursive_compose, strict_recursive_compose2,
lazy_recursive_compose, iterative_compose,
reduce_compose1, reduce_compose2)
print('manual compose', min(timeit.repeat(lambda: manual_compose(5))), manual_compose(5))
for compose in composes:
fn = compose(square, increment, half)
result = min(timeit.repeat(lambda: fn(5)))
print(compose.__name__, result, fn(5))
Results
And we get the following output (same magnitude and proportion in Python 2 and 3):
manual compose 0.4963762479601428 12.25
strict_recursive_compose 0.6564744340721518 12.25
strict_recursive_compose2 0.7216697579715401 12.25
lazy_recursive_compose 1.260614730999805 12.25
iterative_compose 0.614982972969301 12.25
reduce_compose1 0.6768529079854488 12.25
reduce_compose2 0.9890829260693863 12.25
And my expectations were confirmed: the fastest is of course, manual function composition followed by the iterative implementation. The lazy recursive version is much slower - likely since a new stack frame is created by each function call and a new tuple of functions is created for each function.
For a better and perhaps more realistic comparison, if you remove **kwargs
and change *args
to arg
in the functions, the ones that used them will be more performant, and we can better compare apples to apples - here, aside from manual composition, reduce_compose1 wins followed by the strict_recursive_compose:
manual compose 0.443808660027571 12.25
strict_recursive_compose 0.5409777010791004 12.25
strict_recursive_compose2 0.5698030130006373 12.25
lazy_recursive_compose 1.0381018499610946 12.25
iterative_compose 0.619289995986037 12.25
reduce_compose1 0.49532539502251893 12.25
reduce_compose2 0.9633988010464236 12.25
Functions with just one arg:
def strict_recursive_compose(*funcs):
*funcs, penultimate, last = funcs
if funcs:
penultimate = strict_recursive_compose(*funcs, penultimate)
return lambda arg: penultimate(last(arg))
def strict_recursive_compose2(*funcs):
if len(funcs) > 2:
penultimate = strict_recursive_compose2(*funcs[:-1])
else:
penultimate = funcs[-2]
return lambda arg: penultimate(funcs[-1](arg))
def lazy_recursive_compose(*funcs):
def inner(arg, _funcs=funcs):
if len(_funcs) > 1:
return inner(_funcs[-1](arg), _funcs=_funcs[:-1])
else:
return _funcs[0](arg)
return inner
def iterative_compose(*functions):
"""my implementation, only accepts one argument."""
def inner(arg):
for f in reversed(functions):
arg = f(arg)
return arg
return inner
def _compose2(f, g):
return lambda arg: f(g(arg))
def reduce_compose1(*fs):
return reduce(_compose2, fs)
def reduce_compose2(*funcs):
"""bug fixed - added reversed()"""
return lambda x: reduce(lambda acc, f: f(acc), reversed(funcs), x)
Recursive implementation
Here's a fairly elegant recursive implementation, which uses features of Python 3 for clarity:
def strict_compose(*funcs):
*funcs, penultimate, last = funcs
if funcs:
penultimate = strict_compose(*funcs, penultimate)
return lambda *args, **kwargs: penultimate(last(*args, **kwargs))
Python 2 compatible version:
def strict_compose2(*funcs):
if len(funcs) > 2:
penultimate = strict_compose2(*funcs[:-1])
else:
penultimate = funcs[-2]
return lambda *args, **kwargs: penultimate(funcs[-1](*args, **kwargs))
This is an earlier version which uses lazy evaluation of the recursion:
def lazy_recursive_compose(*funcs):
def inner(*args, _funcs=funcs, **kwargs):
if len(_funcs) > 1:
return inner(_funcs[-1](*args, **kwargs), _funcs=_funcs[:-1])
else:
return _funcs[0](*args, **kwargs)
return inner
Both would seem to make a new tuple and dict of arguments each recursive call.
Comparison of all suggestions:
Let's test some of these implementations and determine which is most performant, first some single argument functions (Thank you poke):
def square(x):
return x ** 2
def increment(x):
return x + 1
def half(x):
return x / 2
Here's our implementations, I suspect my iterative version is the second most efficient (manual compose will naturally be fastest), but that may be in part due to it sidestepping the difficulty of passing any number of arguments or keyword arguments between functions - in most cases we'll only see the trivial one argument being passed.
from functools import reduce
def strict_recursive_compose(*funcs):
*funcs, penultimate, last = funcs
if funcs:
penultimate = strict_recursive_compose(*funcs, penultimate)
return lambda *args, **kwargs: penultimate(last(*args, **kwargs))
def strict_recursive_compose2(*funcs):
if len(funcs) > 2:
penultimate = strict_recursive_compose2(*funcs[:-1])
else:
penultimate = funcs[-2]
return lambda *args, **kwargs: penultimate(funcs[-1](*args, **kwargs))
def lazy_recursive_compose(*funcs):
def inner(*args, _funcs=funcs, **kwargs):
if len(_funcs) > 1:
return inner(_funcs[-1](*args, **kwargs), _funcs=_funcs[:-1])
else:
return _funcs[0](*args, **kwargs)
return inner
def iterative_compose(*functions):
"""my implementation, only accepts one argument."""
def inner(arg):
for f in reversed(functions):
arg = f(arg)
return arg
return inner
def _compose2(f, g):
return lambda *a, **kw: f(g(*a, **kw))
def reduce_compose1(*fs):
return reduce(_compose2, fs)
def reduce_compose2(*funcs):
"""bug fixed - added reversed()"""
return lambda x: reduce(lambda acc, f: f(acc), reversed(funcs), x)
And to test these:
import timeit
def manual_compose(n):
return square(increment(half(n)))
composes = (strict_recursive_compose, strict_recursive_compose2,
lazy_recursive_compose, iterative_compose,
reduce_compose1, reduce_compose2)
print('manual compose', min(timeit.repeat(lambda: manual_compose(5))), manual_compose(5))
for compose in composes:
fn = compose(square, increment, half)
result = min(timeit.repeat(lambda: fn(5)))
print(compose.__name__, result, fn(5))
Results
And we get the following output (same magnitude and proportion in Python 2 and 3):
manual compose 0.4963762479601428 12.25
strict_recursive_compose 0.6564744340721518 12.25
strict_recursive_compose2 0.7216697579715401 12.25
lazy_recursive_compose 1.260614730999805 12.25
iterative_compose 0.614982972969301 12.25
reduce_compose1 0.6768529079854488 12.25
reduce_compose2 0.9890829260693863 12.25
And my expectations were confirmed: the fastest is of course, manual function composition followed by the iterative implementation. The lazy recursive version is much slower - likely since a new stack frame is created by each function call and a new tuple of functions is created for each function.
For a better and perhaps more realistic comparison, if you remove **kwargs
and change *args
to arg
in the functions, the ones that used them will be more performant, and we can better compare apples to apples - here, aside from manual composition, reduce_compose1 wins followed by the strict_recursive_compose:
manual compose 0.443808660027571 12.25
strict_recursive_compose 0.5409777010791004 12.25
strict_recursive_compose2 0.5698030130006373 12.25
lazy_recursive_compose 1.0381018499610946 12.25
iterative_compose 0.619289995986037 12.25
reduce_compose1 0.49532539502251893 12.25
reduce_compose2 0.9633988010464236 12.25
Functions with just one arg:
def strict_recursive_compose(*funcs):
*funcs, penultimate, last = funcs
if funcs:
penultimate = strict_recursive_compose(*funcs, penultimate)
return lambda arg: penultimate(last(arg))
def strict_recursive_compose2(*funcs):
if len(funcs) > 2:
penultimate = strict_recursive_compose2(*funcs[:-1])
else:
penultimate = funcs[-2]
return lambda arg: penultimate(funcs[-1](arg))
def lazy_recursive_compose(*funcs):
def inner(arg, _funcs=funcs):
if len(_funcs) > 1:
return inner(_funcs[-1](arg), _funcs=_funcs[:-1])
else:
return _funcs[0](arg)
return inner
def iterative_compose(*functions):
"""my implementation, only accepts one argument."""
def inner(arg):
for f in reversed(functions):
arg = f(arg)
return arg
return inner
def _compose2(f, g):
return lambda arg: f(g(arg))
def reduce_compose1(*fs):
return reduce(_compose2, fs)
def reduce_compose2(*funcs):
"""bug fixed - added reversed()"""
return lambda x: reduce(lambda acc, f: f(acc), reversed(funcs), x)
edited Sep 4 '18 at 12:23
answered Jan 11 '16 at 2:25
Aaron Hall♦Aaron Hall
182k51309262
182k51309262
add a comment |
add a comment |
One liner:
compose = lambda *F: reduce(lambda f, g: lambda x: f(g(x)), F)
Example usage:
f1 = lambda x: x+3
f2 = lambda x: x*2
f3 = lambda x: x-1
g = compose(f1, f2, f3)
assert(g(7) == 15)
add a comment |
One liner:
compose = lambda *F: reduce(lambda f, g: lambda x: f(g(x)), F)
Example usage:
f1 = lambda x: x+3
f2 = lambda x: x*2
f3 = lambda x: x-1
g = compose(f1, f2, f3)
assert(g(7) == 15)
add a comment |
One liner:
compose = lambda *F: reduce(lambda f, g: lambda x: f(g(x)), F)
Example usage:
f1 = lambda x: x+3
f2 = lambda x: x*2
f3 = lambda x: x-1
g = compose(f1, f2, f3)
assert(g(7) == 15)
One liner:
compose = lambda *F: reduce(lambda f, g: lambda x: f(g(x)), F)
Example usage:
f1 = lambda x: x+3
f2 = lambda x: x*2
f3 = lambda x: x-1
g = compose(f1, f2, f3)
assert(g(7) == 15)
edited Jun 1 '16 at 1:00
answered Jun 1 '16 at 0:52
BrettBrett
14113
14113
add a comment |
add a comment |
You can also create an array of functions and use reduce:
def f1(x): return x+1
def f2(x): return x+2
def f3(x): return x+3
x = 5
# Will print f3(f2(f1(x)))
print reduce(lambda acc, x: x(acc), [f1, f2, f3], x)
# As a function:
def compose(*funcs):
return lambda x: reduce(lambda acc, f: f(acc), funcs, x)
f = compose(f1, f2, f3)
Can you show how (/is it even possible) to add an aggregation step - presuming the chained functions are operating on collections?
– javadba
Oct 20 '18 at 21:56
add a comment |
You can also create an array of functions and use reduce:
def f1(x): return x+1
def f2(x): return x+2
def f3(x): return x+3
x = 5
# Will print f3(f2(f1(x)))
print reduce(lambda acc, x: x(acc), [f1, f2, f3], x)
# As a function:
def compose(*funcs):
return lambda x: reduce(lambda acc, f: f(acc), funcs, x)
f = compose(f1, f2, f3)
Can you show how (/is it even possible) to add an aggregation step - presuming the chained functions are operating on collections?
– javadba
Oct 20 '18 at 21:56
add a comment |
You can also create an array of functions and use reduce:
def f1(x): return x+1
def f2(x): return x+2
def f3(x): return x+3
x = 5
# Will print f3(f2(f1(x)))
print reduce(lambda acc, x: x(acc), [f1, f2, f3], x)
# As a function:
def compose(*funcs):
return lambda x: reduce(lambda acc, f: f(acc), funcs, x)
f = compose(f1, f2, f3)
You can also create an array of functions and use reduce:
def f1(x): return x+1
def f2(x): return x+2
def f3(x): return x+3
x = 5
# Will print f3(f2(f1(x)))
print reduce(lambda acc, x: x(acc), [f1, f2, f3], x)
# As a function:
def compose(*funcs):
return lambda x: reduce(lambda acc, f: f(acc), funcs, x)
f = compose(f1, f2, f3)
edited May 24 '13 at 16:36
answered May 24 '13 at 16:30
Imanol LuengoImanol Luengo
10.3k12746
10.3k12746
Can you show how (/is it even possible) to add an aggregation step - presuming the chained functions are operating on collections?
– javadba
Oct 20 '18 at 21:56
add a comment |
Can you show how (/is it even possible) to add an aggregation step - presuming the chained functions are operating on collections?
– javadba
Oct 20 '18 at 21:56
Can you show how (/is it even possible) to add an aggregation step - presuming the chained functions are operating on collections?
– javadba
Oct 20 '18 at 21:56
Can you show how (/is it even possible) to add an aggregation step - presuming the chained functions are operating on collections?
– javadba
Oct 20 '18 at 21:56
add a comment |
Poke's answer is good, but you can also use the functional
package which comes with a compose method.
I was looking into implementing one myself
– Starless
May 24 '13 at 16:32
@Starless It has many recipes. Like any writer, you must read to improve what you write.
– Marcin
May 24 '13 at 16:33
FWIK, functional.compose only supports two arguments.
– georg
May 24 '13 at 17:31
@thg435 It has recipes for a multi-compose.
– Marcin
May 24 '13 at 17:57
@poke solution is convenient in that it takes the arguments ininfix
notation/order
– javadba
Oct 20 '18 at 21:55
add a comment |
Poke's answer is good, but you can also use the functional
package which comes with a compose method.
I was looking into implementing one myself
– Starless
May 24 '13 at 16:32
@Starless It has many recipes. Like any writer, you must read to improve what you write.
– Marcin
May 24 '13 at 16:33
FWIK, functional.compose only supports two arguments.
– georg
May 24 '13 at 17:31
@thg435 It has recipes for a multi-compose.
– Marcin
May 24 '13 at 17:57
@poke solution is convenient in that it takes the arguments ininfix
notation/order
– javadba
Oct 20 '18 at 21:55
add a comment |
Poke's answer is good, but you can also use the functional
package which comes with a compose method.
Poke's answer is good, but you can also use the functional
package which comes with a compose method.
answered May 24 '13 at 16:21
MarcinMarcin
35.7k1289174
35.7k1289174
I was looking into implementing one myself
– Starless
May 24 '13 at 16:32
@Starless It has many recipes. Like any writer, you must read to improve what you write.
– Marcin
May 24 '13 at 16:33
FWIK, functional.compose only supports two arguments.
– georg
May 24 '13 at 17:31
@thg435 It has recipes for a multi-compose.
– Marcin
May 24 '13 at 17:57
@poke solution is convenient in that it takes the arguments ininfix
notation/order
– javadba
Oct 20 '18 at 21:55
add a comment |
I was looking into implementing one myself
– Starless
May 24 '13 at 16:32
@Starless It has many recipes. Like any writer, you must read to improve what you write.
– Marcin
May 24 '13 at 16:33
FWIK, functional.compose only supports two arguments.
– georg
May 24 '13 at 17:31
@thg435 It has recipes for a multi-compose.
– Marcin
May 24 '13 at 17:57
@poke solution is convenient in that it takes the arguments ininfix
notation/order
– javadba
Oct 20 '18 at 21:55
I was looking into implementing one myself
– Starless
May 24 '13 at 16:32
I was looking into implementing one myself
– Starless
May 24 '13 at 16:32
@Starless It has many recipes. Like any writer, you must read to improve what you write.
– Marcin
May 24 '13 at 16:33
@Starless It has many recipes. Like any writer, you must read to improve what you write.
– Marcin
May 24 '13 at 16:33
FWIK, functional.compose only supports two arguments.
– georg
May 24 '13 at 17:31
FWIK, functional.compose only supports two arguments.
– georg
May 24 '13 at 17:31
@thg435 It has recipes for a multi-compose.
– Marcin
May 24 '13 at 17:57
@thg435 It has recipes for a multi-compose.
– Marcin
May 24 '13 at 17:57
@poke solution is convenient in that it takes the arguments in
infix
notation/order– javadba
Oct 20 '18 at 21:55
@poke solution is convenient in that it takes the arguments in
infix
notation/order– javadba
Oct 20 '18 at 21:55
add a comment |
The most reliable implementation I have found is in the 3rd party library toolz
. The compose
function from this library also deals with docstring for the composition of functions.
The source code is freely available. Below is a simple example of usage.
from toolz import compose
def f(x):
return x+1
def g(x):
return x*2
def h(x):
return x+3
res = compose(f, g, h)(5) # 17
add a comment |
The most reliable implementation I have found is in the 3rd party library toolz
. The compose
function from this library also deals with docstring for the composition of functions.
The source code is freely available. Below is a simple example of usage.
from toolz import compose
def f(x):
return x+1
def g(x):
return x*2
def h(x):
return x+3
res = compose(f, g, h)(5) # 17
add a comment |
The most reliable implementation I have found is in the 3rd party library toolz
. The compose
function from this library also deals with docstring for the composition of functions.
The source code is freely available. Below is a simple example of usage.
from toolz import compose
def f(x):
return x+1
def g(x):
return x*2
def h(x):
return x+3
res = compose(f, g, h)(5) # 17
The most reliable implementation I have found is in the 3rd party library toolz
. The compose
function from this library also deals with docstring for the composition of functions.
The source code is freely available. Below is a simple example of usage.
from toolz import compose
def f(x):
return x+1
def g(x):
return x*2
def h(x):
return x+3
res = compose(f, g, h)(5) # 17
answered Apr 28 '18 at 13:14
jppjpp
102k2165116
102k2165116
add a comment |
add a comment |
pip install funcoperators
is another library to implement it that allows infix notation:
from funcoperators import compose
# display = lambda x: hex(ord(list(x)))
display = hex *compose* ord *compose* list
# also works as a function
display = compose(hex, ord, list)
pip install funcoperators https://pypi.org/project/funcoperators/
Disclaimer: I'm the creator of the module
add a comment |
pip install funcoperators
is another library to implement it that allows infix notation:
from funcoperators import compose
# display = lambda x: hex(ord(list(x)))
display = hex *compose* ord *compose* list
# also works as a function
display = compose(hex, ord, list)
pip install funcoperators https://pypi.org/project/funcoperators/
Disclaimer: I'm the creator of the module
add a comment |
pip install funcoperators
is another library to implement it that allows infix notation:
from funcoperators import compose
# display = lambda x: hex(ord(list(x)))
display = hex *compose* ord *compose* list
# also works as a function
display = compose(hex, ord, list)
pip install funcoperators https://pypi.org/project/funcoperators/
Disclaimer: I'm the creator of the module
pip install funcoperators
is another library to implement it that allows infix notation:
from funcoperators import compose
# display = lambda x: hex(ord(list(x)))
display = hex *compose* ord *compose* list
# also works as a function
display = compose(hex, ord, list)
pip install funcoperators https://pypi.org/project/funcoperators/
Disclaimer: I'm the creator of the module
answered Dec 22 '18 at 13:57
Robert Vanden EyndeRobert Vanden Eynde
1059
1059
add a comment |
add a comment |
This is my version
def compose(*fargs):
def inner(arg):
if not arg:
raise ValueError("Invalid argument")
if not all([callable(f) for f in fargs]):
raise TypeError("Function is not callable")
return reduce(lambda arg, func: func(arg), fargs, arg)
return inner
An example of how it's used
def calcMean(iterable):
return sum(iterable) / len(iterable)
def formatMean(mean):
return round(float(mean), 2)
def adder(val, value):
return val + value
def isEven(val):
return val % 2 == 0
if __name__ == '__main__':
# Ex1
rand_range = [random.randint(0, 10000) for x in range(0, 10000)]
isRandIntEven = compose(calcMean, formatMean,
partial(adder, value=0), math.floor.__call__, isEven)
print(isRandIntEven(rand_range))
add a comment |
This is my version
def compose(*fargs):
def inner(arg):
if not arg:
raise ValueError("Invalid argument")
if not all([callable(f) for f in fargs]):
raise TypeError("Function is not callable")
return reduce(lambda arg, func: func(arg), fargs, arg)
return inner
An example of how it's used
def calcMean(iterable):
return sum(iterable) / len(iterable)
def formatMean(mean):
return round(float(mean), 2)
def adder(val, value):
return val + value
def isEven(val):
return val % 2 == 0
if __name__ == '__main__':
# Ex1
rand_range = [random.randint(0, 10000) for x in range(0, 10000)]
isRandIntEven = compose(calcMean, formatMean,
partial(adder, value=0), math.floor.__call__, isEven)
print(isRandIntEven(rand_range))
add a comment |
This is my version
def compose(*fargs):
def inner(arg):
if not arg:
raise ValueError("Invalid argument")
if not all([callable(f) for f in fargs]):
raise TypeError("Function is not callable")
return reduce(lambda arg, func: func(arg), fargs, arg)
return inner
An example of how it's used
def calcMean(iterable):
return sum(iterable) / len(iterable)
def formatMean(mean):
return round(float(mean), 2)
def adder(val, value):
return val + value
def isEven(val):
return val % 2 == 0
if __name__ == '__main__':
# Ex1
rand_range = [random.randint(0, 10000) for x in range(0, 10000)]
isRandIntEven = compose(calcMean, formatMean,
partial(adder, value=0), math.floor.__call__, isEven)
print(isRandIntEven(rand_range))
This is my version
def compose(*fargs):
def inner(arg):
if not arg:
raise ValueError("Invalid argument")
if not all([callable(f) for f in fargs]):
raise TypeError("Function is not callable")
return reduce(lambda arg, func: func(arg), fargs, arg)
return inner
An example of how it's used
def calcMean(iterable):
return sum(iterable) / len(iterable)
def formatMean(mean):
return round(float(mean), 2)
def adder(val, value):
return val + value
def isEven(val):
return val % 2 == 0
if __name__ == '__main__':
# Ex1
rand_range = [random.randint(0, 10000) for x in range(0, 10000)]
isRandIntEven = compose(calcMean, formatMean,
partial(adder, value=0), math.floor.__call__, isEven)
print(isRandIntEven(rand_range))
edited Dec 5 '17 at 0:03
answered Dec 4 '17 at 23:29
CasualCoder3CasualCoder3
17829
17829
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f16739290%2fcomposing-functions-in-python%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown