Dark Mode

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Improve resiliency of JDBC Batch Task on transient failures #796

Open
Improve resiliency of JDBC Batch Task on transient failures#796
Feature
Assignees
Labels
area/pluginPlugin-related issue or feature requestkind/customer-requestRequested by one or more customers

Description

Feature description

The JDBC Batch Task can fail due to transient issues while processing input files (e.g. interrupted file input streams, temporary I/O or network glitches). While we already have ongoing work to mitigate some of these issues (see https://github.com/kestra-io/kestra-ee/issues/6338), the Batch Task itself could be more resilient.

If a transient failure occurs during batch processing, the task fails and requires a full retry from the beginning, even though the failure may be recoverable.

Proposed improvement

Enhance the JDBC Batch Task to:

  • Automatically retry on transient failures (e.g. I/O interruptions, temporary connectivity issues)
  • On retry, refetch the full input file and resume processing from the last successful iteration
  • Avoid retries on non-transient errors, such as database constraint violations or conflicts
  • Make retry behavior configurable with a maximum retry count and sensible default

See also the original Slack thread

Metadata

Metadata

Labels

area/pluginPlugin-related issue or feature requestkind/customer-requestRequested by one or more customers

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions