-
Notifications
You must be signed in to change notification settings - Fork 16.6k
Add asset-based filtering support to DAG API endpoint#54263
Add asset-based filtering support to DAG API endpoint#54263bbovenzi merged 21 commits intoapache:mainfrom
Conversation
Implements two new filter parameters for the DAG API: has_asset_schedule (boolean) to filter DAGs by asset-based scheduling and asset_dependency (string) to filter by asset name/URI using ILIKE pattern matching.
closes: #53740
related: #53041
Testing
Filter with dags with no assets
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.
pierrejeambrun
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice, looks good overall, just a few suggestion, and maybe one thing to test or fix. (join duplicating dags entries)
airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_dags.py
Outdated
Show resolved
Hide resolved
airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_dags.py
Outdated
Show resolved
Hide resolved
pierrejeambrun
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for addressing the comments, just a couple of comments and we should be good to merge.
airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_dags.py
Outdated
Show resolved
Hide resolved
|
@bbovenzi, can you merge this if it looks good to you? |
pierrejeambrun
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just one comment otherwise looking good.
* implement review comments
* fix tests
* fix review comments
* add filters to UI endpoint
* implement review comments
* fix tests
* fix review comments
* add filters to UI endpoint
## New Features:
- Add `map_index` filter to TaskInstance API queries ([#55614](apache/airflow#55614))
- Add `has_import_errors` filter to Core API GET /dags endpoint ([#54563](apache/airflow#54563))
- Add `dag_version` filter to get_dag_runs endpoint ([#54882](apache/airflow#54882))
- Implement pattern search for event log endpoint ([#55114](apache/airflow#55114))
- Add asset-based filtering support to DAG API endpoint ([#54263](apache/airflow#54263))
- Add Greater Than and Less Than range filters to DagRuns and Task Instance list ([#54302](apache/airflow#54302))
- Add `try_number` as filter to task instances ([#54695](apache/airflow#54695))
- Add filters to Browse XComs endpoint ([#54049](apache/airflow#54049))
- Add Filtering by DAG Bundle Name and Version to API routes ([#54004](apache/airflow#54004))
- Add search filter for DAG runs by triggering user name ([#53652](apache/airflow#53652))
- Enable multi sorting (AIP-84) ([#53408](apache/airflow#53408))
- Add `run_on_latest_version` support for backfill and clear operations ([#52177](apache/airflow#52177))
- Add `run_id_pattern` search for Dag Run API ([#52437](apache/airflow#52437))
- Add tracking of triggering user to Dag runs ([#51738](apache/airflow#51738))
- Expose DAG parsing duration in the API ([#54752](apache/airflow#54752))
## New API Endpoints:
- Add Human-in-the-Loop (HITL) endpoints for approval workflows ([#52868](apache/airflow#52868), [#53373](apache/airflow#53373), [#53376](apache/airflow#53376), [#53885](apache/airflow#53885), [#53923](apache/airflow#53923), [#54308](apache/airflow#54308), [#54310](apache/airflow#54310), [#54723](apache/airflow#54723), [#54773](apache/airflow#54773), [#55019](apache/airflow#55019), [#55463](apache/airflow#55463), [#55525](apache/airflow#55525), [#55535](apache/airflow#55535), [#55603](apache/airflow#55603), [#55776](apache/airflow#55776))
- Add endpoint to watch dag run until finish ([#51920](apache/airflow#51920))
- Add TI bulk actions endpoint ([#50443](apache/airflow#50443))
- Add Keycloak Refresh Token Endpoint ([#51657](apache/airflow#51657))
## Deprecations:
- Mark `DagDetailsResponse.concurrency` as deprecated ([#55150](apache/airflow#55150))
## Bug Fixes:
- Fix dag import error modal pagination ([#55719](apache/airflow#55719))
## New Features:
- Add `map_index` filter to TaskInstance API queries ([#55614](apache/airflow#55614))
- Add `has_import_errors` filter to Core API GET /dags endpoint ([#54563](apache/airflow#54563))
- Add `dag_version` filter to get_dag_runs endpoint ([#54882](apache/airflow#54882))
- Implement pattern search for event log endpoint ([#55114](apache/airflow#55114))
- Add asset-based filtering support to DAG API endpoint ([#54263](apache/airflow#54263))
- Add Greater Than and Less Than range filters to DagRuns and Task Instance list ([#54302](apache/airflow#54302))
- Add `try_number` as filter to task instances ([#54695](apache/airflow#54695))
- Add filters to Browse XComs endpoint ([#54049](apache/airflow#54049))
- Add Filtering by DAG Bundle Name and Version to API routes ([#54004](apache/airflow#54004))
- Add search filter for DAG runs by triggering user name ([#53652](apache/airflow#53652))
- Enable multi sorting (AIP-84) ([#53408](apache/airflow#53408))
- Add `run_on_latest_version` support for backfill and clear operations ([#52177](apache/airflow#52177))
- Add `run_id_pattern` search for Dag Run API ([#52437](apache/airflow#52437))
- Add tracking of triggering user to Dag runs ([#51738](apache/airflow#51738))
- Expose DAG parsing duration in the API ([#54752](apache/airflow#54752))
## New API Endpoints:
- Add Human-in-the-Loop (HITL) endpoints for approval workflows ([#52868](apache/airflow#52868), [#53373](apache/airflow#53373), [#53376](apache/airflow#53376), [#53885](apache/airflow#53885), [#53923](apache/airflow#53923), [#54308](apache/airflow#54308), [#54310](apache/airflow#54310), [#54723](apache/airflow#54723), [#54773](apache/airflow#54773), [#55019](apache/airflow#55019), [#55463](apache/airflow#55463), [#55525](apache/airflow#55525), [#55535](apache/airflow#55535), [#55603](apache/airflow#55603), [#55776](apache/airflow#55776))
- Add endpoint to watch dag run until finish ([#51920](apache/airflow#51920))
- Add TI bulk actions endpoint ([#50443](apache/airflow#50443))
- Add Keycloak Refresh Token Endpoint ([#51657](apache/airflow#51657))
## Deprecations:
- Mark `DagDetailsResponse.concurrency` as deprecated ([#55150](apache/airflow#55150))
## Bug Fixes:
- Fix dag import error modal pagination ([#55719](apache/airflow#55719))