-
Notifications
You must be signed in to change notification settings - Fork 16.7k
feat(hitl): include task_instance detail in hitl detail response #53373
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking "Sign up for GitHub", you agree to our terms of service and privacy statement. We'll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Lee-W
merged 4 commits into
apache:main
from
astronomer:add-task-instance-detail-to-hitl-detail-response
Jul 16, 2025
Merged
feat(hitl): include task_instance detail in hitl detail response #53373
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
3dbc149
feat(hitl): include task_instance detail in hitl detail response
Lee-W d79bbbf
fixup! feat(hitl): include task_instance detail in hitl detail response
Lee-W 08afcf6
fixup! fixup! feat(hitl): include task_instance detail in hitl detail...
Lee-W b31186f
fixup! fixup! fixup! feat(hitl): include task_instance detail in hitl...
Lee-W File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -23,6 +23,7 @@ | |
| from pydantic import Field, field_validator | ||
|
|
||
| from airflow.api_fastapi.core_api.base import BaseModel | ||
| from airflow.api_fastapi.core_api.datamodels.task_instances import TaskInstanceResponse | ||
| from airflow.sdk import Param | ||
|
|
||
|
|
||
|
|
@@ -45,7 +46,7 @@ class HITLDetailResponse(BaseModel): | |
| class HITLDetail(BaseModel): | ||
| """Schema for Human-in-the-loop detail.""" | ||
|
|
||
| ti_id: str | ||
| task_instance: TaskInstanceResponse | ||
|
|
||
| # User Request Detail | ||
| options: list[str] | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -9891,9 +9891,8 @@ components: | |
| description: Serializer for Plugin FastAPI root middleware responses. | ||
| HITLDetail: | ||
| properties: | ||
| ti_id: | ||
| type: string | ||
| title: Ti Id | ||
| task_instance: | ||
| $ref: '#/components/schemas/TaskInstanceResponse' | ||
| options: | ||
| items: | ||
| type: string | ||
|
|
@@ -9950,7 +9949,7 @@ components: | |
| default: false | ||
| type: object | ||
| required: | ||
| - ti_id | ||
| - task_instance | ||
| - options | ||
| - subject | ||
| title: HITLDetail | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -19,6 +19,7 @@ | |
| import structlog | ||
| from fastapi import Depends, HTTPException, status | ||
| from sqlalchemy import select | ||
| from sqlalchemy.orm import joinedload | ||
|
|
||
| from airflow.api_fastapi.auth.managers.models.resource_details import DagAccessEntity | ||
| from airflow.api_fastapi.common.db.common import SessionDep, paginated_select | ||
|
|
@@ -132,7 +133,11 @@ def _get_hitl_detail( | |
| ) | ||
|
|
||
| ti_id_str = str(task_instance.id) | ||
| hitl_detail_model = session.scalar(select(HITLDetailModel).where(HITLDetailModel .ti_id == ti_id_str)) | ||
| hitl_detail_model = session.scalar( | ||
| select(HITLDetailModel) | ||
| .where(HITLDetailModel.ti_id == ti_id_str) | ||
| .options(joinedload(HITLDetailModel.task_instance)) | ||
| ) | ||
| if not hitl_detail_model: | ||
| log.error("Human-in-the-loop detail not found") | ||
| raise HTTPException( | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -20,6 +20,7 @@ | |
| from sqlalchemy import Boolean, Column, ForeignKeyConstraint, String, Text | ||
| from sqlalchemy.dialects import postgresql | ||
| from sqlalchemy.ext.hybrid import hybrid_property | ||
| from sqlalchemy.orm import relationship | ||
|
|
||
| from airflow.models.base import Base | ||
| from airflow.settings import json | ||
|
|
@@ -53,6 +54,11 @@ class HITLDetail(Base): | |
| default=None, | ||
| ) | ||
| params_input = Column(sqlalchemy_jsonfield.JSONField(json=json), nullable=False, default={}) | ||
| task_instance = relationship( | ||
| "TaskInstance", | ||
| lazy="joined", | ||
| back_populates="hitl_detail", | ||
| ) | ||
|
|
||
| __table_args__ = ( | ||
| ForeignKeyConstraint( | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -555,6 +555,8 @@ class TaskInstance(Base, LoggingMixin): | |
| triggerer_job = association_proxy("trigger", "triggerer_job") | ||
| dag_run = relationship("DagRun", back_populates="task_instances", lazy="joined", innerjoin=True) | ||
| rendered_task_instance_fields = relationship("RenderedTaskInstanceFields", lazy="noload", uselist=False) | ||
| hitl_detail = relationship("HITLDetail", lazy="noload", uselist=False) | ||
|
|
||
| run_after = association_proxy("dag_run", "run_after") | ||
| logical_date = association_proxy("dag_run", "logical_date") | ||
| task_instance_note = relationship( | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -3410,9 +3410,8 @@ export const $FastAPIRootMiddlewareResponse = { | |
|
|
||
| export const $HITLDetail = { | ||
| properties: { | ||
| ti_id: { | ||
| type: 'string', | ||
| title: 'Ti Id' | ||
| task_instance: { | ||
| '$ref': '#/components/schemas/TaskInstanceResponse' | ||
| }, | ||
| options: { | ||
| items: { | ||
|
|
@@ -3509,7 +3508,7 @@ export const $HITLDetail = { | |
| } | ||
| }, | ||
| type: 'object', | ||
| required: ['ti_id', 'options', 'subject'], | ||
| required: ['task_instance', 'options', 'subject'], | ||
| title: 'HITLDetail', | ||
| description: 'Schema for Human-in-the-loop detail.' | ||
| } as const; | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -921,7 +921,7 @@ export type FastAPIRootMiddlewareResponse = { | |
| * Schema for Human-in-the-loop detail. | ||
| */ | ||
| export type HITLDetail = { | ||
| ti_id: string; | ||
| task_instance: TaskInstanceResponse; | ||
| options: Array<(string)>; | ||
| subject: string; | ||
| body?: string | null; | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -16,6 +16,8 @@ | |
| # under the License. | ||
| from __future__ import annotations | ||
|
|
||
| from unittest import mock | ||
|
|
||
| import pytest | ||
| from sqlalchemy.orm import Session | ||
|
|
||
|
|
@@ -111,8 +113,51 @@ def expected_sample_hitl_detail_dict(sample_ti: TaskInstance) -> dict[str, Any]: | |
| "chosen_options": None, | ||
| "response_received": False, | ||
| "subject": "This is subject", | ||
| "ti_id": sample_ti.id, | ||
| "user_id": None, | ||
| "task_instance": { | ||
| "dag_display_name": "dag", | ||
| "dag_id": "dag", | ||
| "dag_run_id": "test", | ||
| "dag_version": { | ||
| "bundle_name": "dag_maker", | ||
| "bundle_url": None, | ||
| "bundle_version": None, | ||
| "created_at": mock.ANY, | ||
| "dag_display_name": "dag", | ||
| "dag_id": "dag", | ||
| "id": mock.ANY, | ||
| "version_number": 1, | ||
| }, | ||
| "duration": None, | ||
| "end_date": None, | ||
| "executor": None, | ||
| "executor_config": "{}", | ||
| "hostname": "", | ||
| "id": sample_ti.id, | ||
| "logical_date": mock.ANY, | ||
| "map_index": -1, | ||
| "max_tries": 0, | ||
| "note": None, | ||
| "operator": "EmptyOperator", | ||
| "pid": None, | ||
| "pool": "default_pool", | ||
| "pool_slots": 1, | ||
| "priority_weight": 1, | ||
| "queue": "default", | ||
| "queued_when": None, | ||
| "rendered_fields": {}, | ||
| "rendered_map_index": None, | ||
| "run_after": mock.ANY, | ||
| "scheduled_when": None, | ||
| "start_date": None, | ||
| "state": None, | ||
| "task_display_name": "op1", | ||
| "task_id": "op1", | ||
| "trigger": None, | ||
| "triggerer_job": None, | ||
| "try_number": 0, | ||
| "unixname": "root", | ||
| }, | ||
| } | ||
|
|
||
|
|
||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -573,34 +573,6 @@ class FastAPIRootMiddlewareResponse(BaseModel): | |
| name: Annotated[str, Field(title="Name")] | ||
|
|
||
|
|
||
| class HITLDetail(BaseModel): | ||
| """ | ||
| Schema for Human-in-the-loop detail. | ||
| """ | ||
|
|
||
| ti_id: Annotated[str, Field(title="Ti Id")] | ||
| options: Annotated[list[str], Field(title="Options")] | ||
| subject: Annotated[str, Field(title="Subject")] | ||
| body: Annotated[str | None, Field(title="Body")] = None | ||
| defaults: Annotated[list[str] | None, Field(title="Defaults")] = None | ||
| multiple: Annotated[bool | None, Field(title="Multiple")] = False | ||
| params: Annotated[dict[str, Any] | None, Field(title="Params")] = None | ||
| user_id: Annotated[str | None, Field(title="User Id")] = None | ||
| response_at: Annotated[datetime | None, Field(title="Response At")] = None | ||
| chosen_options: Annotated[list[str] | None, Field(title="Chosen Options")] = None | ||
| params_input: Annotated[dict[str, Any] | None, Field(title="Params Input")] = None | ||
| response_received: Annotated[bool | None, Field(title="Response Received")] = False | ||
|
|
||
|
|
||
| class HITLDetailCollection(BaseModel): | ||
| """ | ||
| Schema for a collection of Human-in-the-loop details. | ||
| """ | ||
|
|
||
| hitl_details: Annotated[list[HITLDetail], Field(title="Hitl Details")] | ||
| total_entries: Annotated[int, Field(title="Total Entries")] | ||
|
|
||
|
|
||
| class HITLDetailResponse(BaseModel): | ||
| """ | ||
| Response of updating a Human-in-the-loop detail. | ||
|
|
@@ -1829,6 +1801,34 @@ class DagStatsCollectionResponse(BaseModel): | |
| total_entries: Annotated[int, Field(title="Total Entries")] | ||
|
|
||
|
|
||
| class HITLDetail(BaseModel): | ||
| """ | ||
| Schema for Human-in-the-loop detail. | ||
| """ | ||
|
|
||
| task_instance: TaskInstanceResponse | ||
| options: Annotated[list[str], Field(title="Options")] | ||
| subject: Annotated[str, Field(title="Subject")] | ||
| body: Annotated[str | None, Field(title="Body")] = None | ||
| defaults: Annotated[list[str] | None, Field(title="Defaults")] = None | ||
| multiple: Annotated[bool | None, Field(title="Multiple")] = False | ||
| params: Annotated[dict[str, Any] | None, Field(title="Params")] = None | ||
| user_id: Annotated[str | None, Field(title="User Id")] = None | ||
| response_at: Annotated[datetime | None, Field(title="Response At")] = None | ||
| chosen_options: Annotated[list[str] | None, Field(title="Chosen Options")] = None | ||
| params_input: Annotated[dict[str, Any] | None, Field(title="Params Input")] = None | ||
| response_received: Annotated[bool | None, Field(title="Response Received")] = False | ||
|
|
||
|
|
||
| class HITLDetailCollection(BaseModel): | ||
| """ | ||
| Schema for a collection of Human-in-the-loop details. | ||
| """ | ||
|
|
||
| hitl_details: Annotated[list[HITLDetail], Field(title="Hitl Details")] | ||
| total_entries: Annotated[int, Field(title="Total Entries")] | ||
|
|
||
|
|
||
| class PluginCollectionResponse(BaseModel): | ||
| """ | ||
| Plugin Collection serializer. | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -56,6 +56,7 @@ def compare_attributes(path1, path2): | |
| "triggerer_job", | ||
| "note", | ||
| "rendered_task_instance_fields", | ||
| "hitl_detail", | ||
| # Storing last heartbeat for historic TIs is not interesting/useful | ||
| "last_heartbeat_at", | ||
| "id", | ||
|
|
||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.