generated from kestra-io/plugin-template
-
Notifications
You must be signed in to change notification settings - Fork 32
Description
Feature description
The JDBC Batch Task can fail due to transient issues while processing input files (e.g. interrupted file input streams, temporary I/O or network glitches). While we already have ongoing work to mitigate some of these issues (see https://github.com/kestra-io/kestra-ee/issues/6338), the Batch Task itself could be more resilient.
If a transient failure occurs during batch processing, the task fails and requires a full retry from the beginning, even though the failure may be recoverable.
Proposed improvement
Enhance the JDBC Batch Task to:
- Automatically retry on transient failures (e.g. I/O interruptions, temporary connectivity issues)
- On retry, refetch the full input file and resume processing from the last successful iteration
- Avoid retries on non-transient errors, such as database constraint violations or conflicts
- Make retry behavior configurable with a maximum retry count and sensible default
See also the original Slack thread