-
Notifications
You must be signed in to change notification settings - Fork 16.7k
Skip PodGenerator import for deserialization when no k8s installed#56733
Merged
kaxil merged 3 commits intoapache:mainfrom Oct 16, 2025
Merged
Skip PodGenerator import for deserialization when no k8s installed#56733kaxil merged 3 commits intoapache:mainfrom
kaxil merged 3 commits intoapache:mainfrom
Conversation
Member
The #56692 introduced optimization for PodGenerator imports - but there was a problem that when deserializing Pod it failed when no k8s classes were loaded - but it really is not optimisation but failure - nothing actually prevents us from importing the k8s classes and we actually have to do it in case we want to deserialize serialized Pod.
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.
potiuk
added
full tests needed
We need to run full set of tests for this PR to merge
backport-to-v3-1-test
Mark PR with this label to backport to v3-1-test branch
labels
Oct 16, 2025
kaxil
approved these changes
Oct 16, 2025
amoghrajesh
approved these changes
Oct 16, 2025
gopidesupavan
approved these changes
Oct 16, 2025
This was referenced Oct 16, 2025
potiuk
force-pushed
the
fix-k8s-deserialization
branch
from
October 16, 2025 17:41
b0c5eb2 to
6365b18
Compare
kaxil
reviewed
Oct 16, 2025
The apache#56692 introduced optimization for PodGenerator imports - but there
was a problem that when deserializing Pod it failed when no k8s classes
were loaded - but it really is not optimisation but failure - nothing
actually prevents us from importing the k8s classes and we actually have
to do it in case we want to deserialize serialized Pod. # Please enter
the commit message for your changes. Lines starting
was a problem that when deserializing Pod it failed when no k8s classes
were loaded - but it really is not optimisation but failure - nothing
actually prevents us from importing the k8s classes and we actually have
to do it in case we want to deserialize serialized Pod. # Please enter
the commit message for your changes. Lines starting
potiuk
force-pushed
the
fix-k8s-deserialization
branch
from
October 16, 2025 17:56
6365b18 to
78407dd
Compare
potiuk
changed the title
Lazy import PodGenerator for deserialization
Skip PodGenerator import for deserialization when no k8s installed
Oct 16, 2025
Member
|
The log failures seems to be a different issue: .
E AssertionError: None of the log sources ['/opt/airflow/logs/dag_id=example_xcom_test/run_id=manual__ 2025-10-16T19:00:48.103160+00:00/task_id=bash_pull/attempt=1 .log'] were found in S3 bucket logs ['s3://test-airflow-logs/dag_id=example_xcom_test/run_id=man ual__2025-10-16T19:00:48.103160+00:00/task_id=bash_push/atte mpt=1.log', 's3://test-airflow-logs/dag_id=example_xcom_test/run_id=manu al__2025-10-16T19:00:48.103160+00:00/task_id=pull_value_from _bash_push/attempt=1.log', 's3://test-airflow-logs/dag_id=example_xcom_test/run_id=manu al__2025-10-16T19:00:48.103160+00:00/task_id=puller/attempt= 1.log', 's3://test-airflow-logs/dag_id=example_xcom_test/run_id=manu al__2025-10-16T19:00:48.103160+00:00/task_id=push/attempt=1. log', 's3://test-airflow-logs/dag_id=example_xcom_test/run_id=manual__2025-10-16T19:00:48.103160+00:00/task_id=push_by_returning/attempt=1.log']cc @gopidesupavan if you have ideas |
Member
Author
|
Yeah. Just looked :) |
Backport failed to create: v3-1-test. View the failure log Run details
You can attempt to backport this manually by running: cherry_picker 17037e6 v3-1-test
This should apply the commit to the v3-1-test branch and leave the commit in conflict state marking After you have resolved the conflicts, you can continue the backport process by running: cherry_picker --continue
|
Member
huh thats strange looks like logs endpoint returning local log location instead of remote location, though we have already configured remote logging. Will take a look. |
abdulrahman305 bot
pushed a commit
to qenex-ai/airflow
that referenced
this pull request
Oct 17, 2025
The apache#56692 introduced optimization for PodGenerator imports - but there
was a problem that when deserializing Pod it failed when no k8s classes
were loaded - but it really is not optimisation but failure - nothing
actually prevents us from importing the k8s classes and we actually have
to do it in case we want to deserialize serialized Pod. # Please enter
the commit message for your changes. Lines starting
* fixup! Skip PodGenerator import for deserialization when no k8s installed
* fixup! fixup! Skip PodGenerator import for deserialization when no k8s installed
---------
Co-authored-by: Kaxil Naik
was a problem that when deserializing Pod it failed when no k8s classes
were loaded - but it really is not optimisation but failure - nothing
actually prevents us from importing the k8s classes and we actually have
to do it in case we want to deserialize serialized Pod. # Please enter
the commit message for your changes. Lines starting
* fixup! Skip PodGenerator import for deserialization when no k8s installed
* fixup! fixup! Skip PodGenerator import for deserialization when no k8s installed
---------
Co-authored-by: Kaxil Naik
abdulrahman305 bot
pushed a commit
to qenex-ai/airflow
that referenced
this pull request
Oct 19, 2025
The apache#56692 introduced optimization for PodGenerator imports - but there
was a problem that when deserializing Pod it failed when no k8s classes
were loaded - but it really is not optimisation but failure - nothing
actually prevents us from importing the k8s classes and we actually have
to do it in case we want to deserialize serialized Pod. # Please enter
the commit message for your changes. Lines starting
* fixup! Skip PodGenerator import for deserialization when no k8s installed
* fixup! fixup! Skip PodGenerator import for deserialization when no k8s installed
---------
Co-authored-by: Kaxil Naik
was a problem that when deserializing Pod it failed when no k8s classes
were loaded - but it really is not optimisation but failure - nothing
actually prevents us from importing the k8s classes and we actually have
to do it in case we want to deserialize serialized Pod. # Please enter
the commit message for your changes. Lines starting
* fixup! Skip PodGenerator import for deserialization when no k8s installed
* fixup! fixup! Skip PodGenerator import for deserialization when no k8s installed
---------
Co-authored-by: Kaxil Naik
kaxil
added a commit
that referenced
this pull request
Oct 21, 2025
The #56692 introduced optimization for PodGenerator imports - but there
was a problem that when deserializing Pod it failed when no k8s classes
were loaded - but it really is not optimisation but failure - nothing
actually prevents us from importing the k8s classes and we actually have
to do it in case we want to deserialize serialized Pod. # Please enter
the commit message for your changes. Lines starting
* fixup! Skip PodGenerator import for deserialization when no k8s installed
* fixup! fixup! Skip PodGenerator import for deserialization when no k8s installed
---------
Co-authored-by: Kaxil Naik
(cherry picked from commit 17037e6)
was a problem that when deserializing Pod it failed when no k8s classes
were loaded - but it really is not optimisation but failure - nothing
actually prevents us from importing the k8s classes and we actually have
to do it in case we want to deserialize serialized Pod. # Please enter
the commit message for your changes. Lines starting
* fixup! Skip PodGenerator import for deserialization when no k8s installed
* fixup! fixup! Skip PodGenerator import for deserialization when no k8s installed
---------
Co-authored-by: Kaxil Naik
(cherry picked from commit 17037e6)
TyrellHaywood
pushed a commit
to TyrellHaywood/airflow
that referenced
this pull request
Oct 22, 2025
The apache#56692 introduced optimization for PodGenerator imports - but there
was a problem that when deserializing Pod it failed when no k8s classes
were loaded - but it really is not optimisation but failure - nothing
actually prevents us from importing the k8s classes and we actually have
to do it in case we want to deserialize serialized Pod. # Please enter
the commit message for your changes. Lines starting
* fixup! Skip PodGenerator import for deserialization when no k8s installed
* fixup! fixup! Skip PodGenerator import for deserialization when no k8s installed
---------
Co-authored-by: Kaxil Naik
was a problem that when deserializing Pod it failed when no k8s classes
were loaded - but it really is not optimisation but failure - nothing
actually prevents us from importing the k8s classes and we actually have
to do it in case we want to deserialize serialized Pod. # Please enter
the commit message for your changes. Lines starting
* fixup! Skip PodGenerator import for deserialization when no k8s installed
* fixup! fixup! Skip PodGenerator import for deserialization when no k8s installed
---------
Co-authored-by: Kaxil Naik
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.