[go: up one dir, main page]
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Task timeout support #160

Open
kbakk opened this issue Apr 13, 2021 · 2 comments
Open

Task timeout support #160

kbakk opened this issue Apr 13, 2021 · 2 comments
Labels
enhancement New feature or request

Comments

@kbakk
Copy link
Collaborator
kbakk commented Apr 13, 2021

Is your feature request related to a problem? Please describe.
If a task takes longer than 10 seconds to complete (by default, can be changed), The Zeebe broker will mark the task as timed out. However, the task may continue to progress on the worker. In this case, as the broker will allow other workers to start the task, it will lead to the task being processed twice.

To prevent this, having a way to cancel the task (e.g. with SIGINT), would prevent the task from being worked on by multiple instances.

Describe the solution you'd like
ZeebeWorker could get an argument cancel_on_timeout that means that the task will be cancelled if it runs longer than the timeout allows.

Describe alternatives you've considered
The user could also implement such a functionality within the task. However, it's not clear what should happen in case it times out, as failing the task (letting an exception bubble up) would mark the task as failed on the broker. Maybe the fact that the worker should allow the task to time out (i.e. not send anything back to the broker) could be indicated with a TimeOutException being raised in the user's code.
@Chadys
Copy link
Contributor
Chadys commented Apr 21, 2021 •
edited
Loading

I prefer the alternative you provided, it makes the solution more generic since it can also be used for tasks that shouldn't be stopped by external code. The tasks needs to be aware of the "at least once" policy of Zeebe anyway to deal with idempotency and such, so making them responsible of correctly reacting to timeouts seams not to be such a stretch. Might be great however to add a very simple way for a task to check "have I timed out?".
We should probably also add a "Timeout" to pyzeebe.JobStatus enum.
From the documentation, it's not clear whether a timed out task that fail has any impact (it only says that a success is not relevant any more), so depending on that a special treatment by Pyzeebe of the TimeOutException will be needed or not (but I like the idea of an explicit exception in any case).

@kbakk kbakk added the enhancement New feature or request label Apr 23, 2021
@kbakk
Copy link
Collaborator Author
kbakk commented Jun 4, 2021

There seems to be a feature in asyncio that can be used for this: https://docs.python.org/3/library/asyncio-task.html#timeouts

Relates to #143

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Chadys @kbakk