Allow MSGraphAsyncOperator and MSGraphSensor to directly start from trigger instead of worker#53009
Allow MSGraphAsyncOperator and MSGraphSensor to directly start from trigger instead of worker#53009dabla wants to merge 33 commits intoapache:mainfrom
Conversation
…art from trigger instead of worker
providers/microsoft/azure/src/airflow/providers/microsoft/azure/operators/msgraph.py
Outdated
Show resolved
Hide resolved
providers/microsoft/azure/src/airflow/providers/microsoft/azure/operators/msgraph.py
Outdated
Show resolved
Hide resolved
… start from trigger
|
I think there is an issue with the defer_task method of TaskInstance, it should call render_template_fields first before calling the expand_start_trigger_args method. Otherwise non rendered fields would be passed to the triggerer to instantiate, which is a not the same behaviour as with regular operators defering afterwards. |
|
Ok overriding the expand_start_trigger_args doesn't help because the scheduler has a serialized version of the task (e.g. operator), so it won't take into account that overriden method. So now remains the question, how do we fix this issue. This is clearly a situation which hasn't been thought about. |
|
This PR must be merged before this one can. |
… is 3.1 or higher as it will contain fix for templated fields
…er instead of expand_start_from_trigger
…stead of expand_start_trigger_args
…_args instead of expand_start_trigger_args" This reverts commit f51cd7b.
|
Created new PR which attempts to fix template rendering with start from trigger, as long as template rendering in triggerer isn't fixed, this PR can't be merged. |
|
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions. |
Allow MSGraphAsyncOperator and MSGraphSensor to always start from trigger instead of worker as this avoids unnecessary round trips to the scheduler.
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in airflow-core/newsfragments.