Backfill jobs

Last updated 10/05/2026

Overview

Backfill Jobs is a feature in the AE DataOps Platform for handling missing or updated data in specific scenarios. When data for a historical time range is incompletely calculated or produces incorrect results because of a system failure, a program exception, a business logic change, or another reason, a backfill job can reprocess and batch backfill the data for that time range.

By configuring a backfill strategy, you can precisely trace back data pipelines and efficiently rerun Flows, so that the data results for the specified time range are consistent with the latest logic. This ensures the completeness, accuracy, and consistency of data in your data warehouse or data lake, and provides key support for the continuity and reliability of business analysis.

Create a backfill job

On the Ops → Backfill Jobs page, click + Create Backfill Job in the upper-right corner to open the creation page

Backfill job settings

Field	Description
Backfill Job Name	Unique within the space. Can contain lowercase letters, numbers, and underscores, and can't exceed 80 characters
Related Flow	You can select an "Online" Flow as the "source Flow" for the backfill job
Backfill Method	Whole flow: Runs all node tasks in the selected Flow
	Node only: Only this node's task in the selected Flow is in the task scope of this backfill
	To node: Runs from the start node of the Flow to this node (inclusive)
	From node: Runs from this node (inclusive) to the end node of the Flow
View DAG	The system automatically parses and highlights the DAG based on "Select Node", so you can view the DAG relationships of the Flow

Backfill job strategy

In Backfill Job Strategy, you can select the range of backfill base dates and the run rules of the job.

Field	Description
Base date ${bd}: The base date for data settlement, and the unit for selecting backfill dates
Backfill Time Range	You can select a date range for the backfill. Each ${bd} in the range generates a backfill Flow instance ${bd} supports selecting future dates The ${bd} backfill date range can't exceed 2 years
Date Setting Method	By Step Step Unit: Determines the step granularity. Defaults to day; supports day/week/month Step Interval: The length of the interval between backfill dates, in date units. Defaults to 1 Manual Selection Supports manually selecting multiple separate ${bd} execution dates
Order	Ascending Base Date Descending Base Date
Run in Parallel	Default: On (Yes) When parallel running is off, the job runs serially in the order of the specified business dates by default When parallel running is on, the business dates are split by the number of groups, and multiple backfill batches are generated from the groups and run concurrently. The number of groups defaults to 2
Exception handling strategy	If backfill job instance fails Job Fail (default) Job Continue

Backfill job statuses

Backfill job status	Description	Available operations
To be submitted	Draft status. Not yet submitted for execution, with no run instances	Edit Delete Manual Execution
Running	After the job is submitted to run, its status becomes "Running"	Terminate
Finished	The backfill job ran to completion successfully	Delete
Failed	If the Flow instance for any base date ${bd} fails to run, the backfill job is considered failed	Rerun Delete
Terminating	Transitional status while the job is being terminated	-
Terminated	If you click "Terminate" while the job is running, the backfill job is terminated	Rerun

Backfill job management

On the Ops → Backfill Jobs page, you can manage all backfill jobs in the space. On the list page, you can view how backfill jobs ran, rerun jobs, manually terminate running jobs, and more.

The management fields are as follows:

Field	Description
Backfill Job ID	Auto-increment unique ID
Backfill Job Name	Backfill Job Name
Backfill Method	Supported: Whole flow, Node only, To node, From node
Status	Includes 6 statuses: To be submitted, Finished, Running, Terminating, Terminated, Failed
Time Cost	Total run time of all latest Flow instances under the job (including queuing time)
Owner	Creator of the backfill job
Related Flow	The "source Flow" associated with the current backfill job
Created at	Creation time of the backfill job
Start Time	Time when the latest Flow instance under the backfill job started running
End Time	Time when the last Flow instance under the backfill job finished running
Last Modified Time	Time of the last edit and save Note: After a backfill job is run manually, it can't be edited again
Action	Edit, run, delete, rerun, and more
Progress	Execution progress of the backfill job