manage early return of event loop with python
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ height:90px;width:728px;box-sizing:border-box;
}
I have a service running the following loop
while True:
feedback = f1()
if check1(feedback):
break
feedback = f2()
if check2(feedback):
break
feedback = f3()
if check3(feedback):
break
time.sleep(10)
do_cleanup(feedback)
Now I would like to run these feedback checks with different time intervals. One naive way is to move the time.sleep() into the f functions. But that causes blocking. What would be the easiest way to achieve periodic checks with different intervals? Here all the f functions are cheap to run.
The event loop in asyncio sounds like the way to go. But due to my inexperience, I don't know where the check and break logic should go for the event loop.
Or is there any other packages/code patterns to do this kind of monitoring logic?
python-3.x monitoring python-asyncio
add a comment |
I have a service running the following loop
while True:
feedback = f1()
if check1(feedback):
break
feedback = f2()
if check2(feedback):
break
feedback = f3()
if check3(feedback):
break
time.sleep(10)
do_cleanup(feedback)
Now I would like to run these feedback checks with different time intervals. One naive way is to move the time.sleep() into the f functions. But that causes blocking. What would be the easiest way to achieve periodic checks with different intervals? Here all the f functions are cheap to run.
The event loop in asyncio sounds like the way to go. But due to my inexperience, I don't know where the check and break logic should go for the event loop.
Or is there any other packages/code patterns to do this kind of monitoring logic?
python-3.x monitoring python-asyncio
add a comment |
I have a service running the following loop
while True:
feedback = f1()
if check1(feedback):
break
feedback = f2()
if check2(feedback):
break
feedback = f3()
if check3(feedback):
break
time.sleep(10)
do_cleanup(feedback)
Now I would like to run these feedback checks with different time intervals. One naive way is to move the time.sleep() into the f functions. But that causes blocking. What would be the easiest way to achieve periodic checks with different intervals? Here all the f functions are cheap to run.
The event loop in asyncio sounds like the way to go. But due to my inexperience, I don't know where the check and break logic should go for the event loop.
Or is there any other packages/code patterns to do this kind of monitoring logic?
python-3.x monitoring python-asyncio
I have a service running the following loop
while True:
feedback = f1()
if check1(feedback):
break
feedback = f2()
if check2(feedback):
break
feedback = f3()
if check3(feedback):
break
time.sleep(10)
do_cleanup(feedback)
Now I would like to run these feedback checks with different time intervals. One naive way is to move the time.sleep() into the f functions. But that causes blocking. What would be the easiest way to achieve periodic checks with different intervals? Here all the f functions are cheap to run.
The event loop in asyncio sounds like the way to go. But due to my inexperience, I don't know where the check and break logic should go for the event loop.
Or is there any other packages/code patterns to do this kind of monitoring logic?
python-3.x monitoring python-asyncio
python-3.x monitoring python-asyncio
edited Nov 26 '18 at 21:40
nos
asked Nov 26 '18 at 21:26
nosnos
6,934145581
6,934145581
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
In asyncio you might split the service into three separate tasks, each with its own loop and timing - you can think of them as three threads, except they are all scheduled in the same thread, and multi-task cooperatively by suspending at await.
For this purpose let's start with a utility function that calls a function and checks its result at a regular interval:
async def at_interval(f, check, seconds):
while True:
feedback = f()
if check(feedback):
return feedback
await asyncio.sleep(seconds)
The return is the equivalent to the break in your original code.
With that in place, the service spawns three such loops and wait for any of them to finish. Whichever completes first carries the "feedback" we're waiting for, and we can dispose of the others.
async def service():
loop = asyncio.get_event_loop()
t1 = loop.create_task(at_interval(f1, check1, 3))
t2 = loop.create_task(at_interval(f2, check2, 5))
t3 = loop.create_task(at_interval(f3, check3, 7))
done, pending = await asyncio.wait(
[t1, t2, t3], return_when=asyncio.FIRST_COMPLETED)
for t in pending:
t.cancel()
feedback = await list(done)[0]
do_cleanup(feedback)
asyncio.get_event_loop().run_until_complete(service())
A small difference between this and your code is that here it is possible (though very unlikely) for more than one check to fail before the service picks up on it. For example, if through a stroke of bad luck two of the above tasks end up sharing the absolute time of wakeup to the microsecond, they will be scheduled in the same event loop iteration. Both will return from their corresponding at_interval coroutines, and done will contain more than one feedback. The code handles it by picking a feedback and calling do_cleanup on that one, but it could also loop over all.
If this is not acceptable, you can easily pass each at_interval a callable that cancels all tasks except itself. This is currently done in service for brevity, but it can be done in at_interval as well. One task cancelling the others would ensure that only one feedback can exist.
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53489316%2fmanage-early-return-of-event-loop-with-python%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
In asyncio you might split the service into three separate tasks, each with its own loop and timing - you can think of them as three threads, except they are all scheduled in the same thread, and multi-task cooperatively by suspending at await.
For this purpose let's start with a utility function that calls a function and checks its result at a regular interval:
async def at_interval(f, check, seconds):
while True:
feedback = f()
if check(feedback):
return feedback
await asyncio.sleep(seconds)
The return is the equivalent to the break in your original code.
With that in place, the service spawns three such loops and wait for any of them to finish. Whichever completes first carries the "feedback" we're waiting for, and we can dispose of the others.
async def service():
loop = asyncio.get_event_loop()
t1 = loop.create_task(at_interval(f1, check1, 3))
t2 = loop.create_task(at_interval(f2, check2, 5))
t3 = loop.create_task(at_interval(f3, check3, 7))
done, pending = await asyncio.wait(
[t1, t2, t3], return_when=asyncio.FIRST_COMPLETED)
for t in pending:
t.cancel()
feedback = await list(done)[0]
do_cleanup(feedback)
asyncio.get_event_loop().run_until_complete(service())
A small difference between this and your code is that here it is possible (though very unlikely) for more than one check to fail before the service picks up on it. For example, if through a stroke of bad luck two of the above tasks end up sharing the absolute time of wakeup to the microsecond, they will be scheduled in the same event loop iteration. Both will return from their corresponding at_interval coroutines, and done will contain more than one feedback. The code handles it by picking a feedback and calling do_cleanup on that one, but it could also loop over all.
If this is not acceptable, you can easily pass each at_interval a callable that cancels all tasks except itself. This is currently done in service for brevity, but it can be done in at_interval as well. One task cancelling the others would ensure that only one feedback can exist.
add a comment |
In asyncio you might split the service into three separate tasks, each with its own loop and timing - you can think of them as three threads, except they are all scheduled in the same thread, and multi-task cooperatively by suspending at await.
For this purpose let's start with a utility function that calls a function and checks its result at a regular interval:
async def at_interval(f, check, seconds):
while True:
feedback = f()
if check(feedback):
return feedback
await asyncio.sleep(seconds)
The return is the equivalent to the break in your original code.
With that in place, the service spawns three such loops and wait for any of them to finish. Whichever completes first carries the "feedback" we're waiting for, and we can dispose of the others.
async def service():
loop = asyncio.get_event_loop()
t1 = loop.create_task(at_interval(f1, check1, 3))
t2 = loop.create_task(at_interval(f2, check2, 5))
t3 = loop.create_task(at_interval(f3, check3, 7))
done, pending = await asyncio.wait(
[t1, t2, t3], return_when=asyncio.FIRST_COMPLETED)
for t in pending:
t.cancel()
feedback = await list(done)[0]
do_cleanup(feedback)
asyncio.get_event_loop().run_until_complete(service())
A small difference between this and your code is that here it is possible (though very unlikely) for more than one check to fail before the service picks up on it. For example, if through a stroke of bad luck two of the above tasks end up sharing the absolute time of wakeup to the microsecond, they will be scheduled in the same event loop iteration. Both will return from their corresponding at_interval coroutines, and done will contain more than one feedback. The code handles it by picking a feedback and calling do_cleanup on that one, but it could also loop over all.
If this is not acceptable, you can easily pass each at_interval a callable that cancels all tasks except itself. This is currently done in service for brevity, but it can be done in at_interval as well. One task cancelling the others would ensure that only one feedback can exist.
add a comment |
In asyncio you might split the service into three separate tasks, each with its own loop and timing - you can think of them as three threads, except they are all scheduled in the same thread, and multi-task cooperatively by suspending at await.
For this purpose let's start with a utility function that calls a function and checks its result at a regular interval:
async def at_interval(f, check, seconds):
while True:
feedback = f()
if check(feedback):
return feedback
await asyncio.sleep(seconds)
The return is the equivalent to the break in your original code.
With that in place, the service spawns three such loops and wait for any of them to finish. Whichever completes first carries the "feedback" we're waiting for, and we can dispose of the others.
async def service():
loop = asyncio.get_event_loop()
t1 = loop.create_task(at_interval(f1, check1, 3))
t2 = loop.create_task(at_interval(f2, check2, 5))
t3 = loop.create_task(at_interval(f3, check3, 7))
done, pending = await asyncio.wait(
[t1, t2, t3], return_when=asyncio.FIRST_COMPLETED)
for t in pending:
t.cancel()
feedback = await list(done)[0]
do_cleanup(feedback)
asyncio.get_event_loop().run_until_complete(service())
A small difference between this and your code is that here it is possible (though very unlikely) for more than one check to fail before the service picks up on it. For example, if through a stroke of bad luck two of the above tasks end up sharing the absolute time of wakeup to the microsecond, they will be scheduled in the same event loop iteration. Both will return from their corresponding at_interval coroutines, and done will contain more than one feedback. The code handles it by picking a feedback and calling do_cleanup on that one, but it could also loop over all.
If this is not acceptable, you can easily pass each at_interval a callable that cancels all tasks except itself. This is currently done in service for brevity, but it can be done in at_interval as well. One task cancelling the others would ensure that only one feedback can exist.
In asyncio you might split the service into three separate tasks, each with its own loop and timing - you can think of them as three threads, except they are all scheduled in the same thread, and multi-task cooperatively by suspending at await.
For this purpose let's start with a utility function that calls a function and checks its result at a regular interval:
async def at_interval(f, check, seconds):
while True:
feedback = f()
if check(feedback):
return feedback
await asyncio.sleep(seconds)
The return is the equivalent to the break in your original code.
With that in place, the service spawns three such loops and wait for any of them to finish. Whichever completes first carries the "feedback" we're waiting for, and we can dispose of the others.
async def service():
loop = asyncio.get_event_loop()
t1 = loop.create_task(at_interval(f1, check1, 3))
t2 = loop.create_task(at_interval(f2, check2, 5))
t3 = loop.create_task(at_interval(f3, check3, 7))
done, pending = await asyncio.wait(
[t1, t2, t3], return_when=asyncio.FIRST_COMPLETED)
for t in pending:
t.cancel()
feedback = await list(done)[0]
do_cleanup(feedback)
asyncio.get_event_loop().run_until_complete(service())
A small difference between this and your code is that here it is possible (though very unlikely) for more than one check to fail before the service picks up on it. For example, if through a stroke of bad luck two of the above tasks end up sharing the absolute time of wakeup to the microsecond, they will be scheduled in the same event loop iteration. Both will return from their corresponding at_interval coroutines, and done will contain more than one feedback. The code handles it by picking a feedback and calling do_cleanup on that one, but it could also loop over all.
If this is not acceptable, you can easily pass each at_interval a callable that cancels all tasks except itself. This is currently done in service for brevity, but it can be done in at_interval as well. One task cancelling the others would ensure that only one feedback can exist.
edited Nov 26 '18 at 22:30
answered Nov 26 '18 at 21:42
user4815162342user4815162342
64.8k596151
64.8k596151
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53489316%2fmanage-early-return-of-event-loop-with-python%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown