-
Notifications
You must be signed in to change notification settings - Fork 16.6k
AIP-86 - Implement Deadline Callback definition#53951
Conversation
This adds Callback classes for each type of currently planned callback. This will require the user to specify what kind of callback it is (async/sync) and how the callback should be run (kwargs, specific executor etc). In the previous implementation in which the callback type would detected automatically, there were some edge cases that would require a complex solution to handle: primarily the case where a sync callback is only present on the executor where it needs to run, limiting the scheduler's ability to figure out where to send it to be run.
I also changed the database schema to store the serialized version of the callback.
Resolves #44000
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.
|
|
846938c to
3ca4c2a
Compare
callback. This will require the user to specify what kind of callback it
is (async/sync) and how the callback should be run (kwargs, specific
executor etc). In the previous implementation in which the callback type
would detected automatically, there were some edge cases that would
require a complex solution to handle: primarily the case where a sync
callback is only present on the executor where it needs to run, limiting
the scheduler's ability to figure out where to send it to be run.
I also changed the database schema to store the serialized version of
the callback.
3ca4c2a to
fdd1c6f
Compare
aa3b4bb to
cc05e7c
Compare
ferruzzi
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some nits and comments but nothing major.
a2b7af6 to
c230dd2
Compare
This adds `Callback` classes for each type of currently planned
callback. This will require the user to specify what kind of callback it
is (async/sync) and how the callback should be run (kwargs, specific
executor etc). In the previous implementation in which the callback type
would detected automatically, there were some edge cases that would
require a complex solution to handle: primarily the case where a sync
callback is only present on the executor where it needs to run, limiting
the scheduler's ability to figure out where to send it to be run.
I also changed the database schema to store the serialized version of
the callback.
This adds `Callback` classes for each type of currently planned
callback. This will require the user to specify what kind of callback it
is (async/sync) and how the callback should be run (kwargs, specific
executor etc). In the previous implementation in which the callback type
would detected automatically, there were some edge cases that would
require a complex solution to handle: primarily the case where a sync
callback is only present on the executor where it needs to run, limiting
the scheduler's ability to figure out where to send it to be run.
I also changed the database schema to store the serialized version of
the callback.