Flow management and operations
Flow list management
Click the tab to switch to the task node list, which shows details of the task nodes in all Flows in the current workspace. On this tab, you can quickly search for and locate the nodes you're interested in, which makes troubleshooting, data validation, and similar work much more efficient.
The columns of the task node list are as follows:
| Column name | Description |
|---|---|
| Task Node Name | The name of the task node. Click it to open the Flow's detail page with the node's information panel open |
| Belonging Flow | The name of the Flow the task node belongs to. Click it to open the Flow's detail page |
| Type | The type of the task node. Filterable |
| Status | The release status of the task node, such as Pending Releasing, Releasing, and Released. Filterable |
| Owner | The owner of the task node |
| Last Trigger Time | The time of the task node's last scheduled or manual run |
| Scheduling Status | The status of the task node's last scheduled run, such as Pending Submission, Waiting, Running, Retry Waiting, Finished, Failed, Terminated, and Skip. Filterable |
| Last Modified Time | The time the task node was last modified |
| Last Modifier | The user who last modified the task node |
| Created at | The time the task node was created |
| Creator | The creator of the task node |
Finally, click the Release tab to switch to the release list. This tab is described in detail in Flow release
Flow configuration management
DAG area
The DAG area shows the overall structure of the Flow, including task nodes and the relationships between them. You can adjust node positions directly on the DAG, or add dependencies between task nodes by dragging.
Hover over a task node to see the operations available for it:
- Click and drag from the small dots around a node to add a dependency between task nodes
- To expand a node's detail panel, click View details, select View details in the ··· menu, or double-click the task node. In this panel, you can view the node's details, debug it, or enter edit mode to edit its content.
- In the ··· menu, you can rename the task node (only if the node hasn't been released) or delete the selected node
- After you click a node to select it, click the + button that appears on its right to quickly add a new node after it. The new node automatically depends on the current node
You can also adjust the DAG canvas and node layout from the DAG toolbar.
- Navigation area
The navigation area provides a set of operations available in Dev Mode:
- Production Mode: select Production Mode from the mode drop-down at the top to switch to Production Mode. The Flow must have been released (this option is grayed out if it hasn't)
- Delete: in the ··· menu in the upper-right corner (the same menu also has Edit basic info and Add My Subscription). Deletes the Flow; the Flow must not have been released. To delete a released Flow, first take it offline in Production Mode, then delete it
- Release: releases the changes in the Flow. Available only when the Flow has changes or hasn't been released. Clicking it opens the new release page
Task configuration
In the task configuration panel, you can set the node's owner and execution policies. In addition, for nodes that write data, such as Offline sync plan nodes and Trino SQL nodes, you can view and manually parse the workspace tables the task node writes to (they're also parsed automatically when you save).
Execution policies include Skip Execution and Retry Strategy:
- Skip Execution: when this switch is on, the node is skipped in both manual and scheduled runs, and no task node instance is created. The node is also skipped during debugging
- Retry Strategy: when this switch is on, the task node retries automatically after a failed run, and each retry creates a new task node instance. If the node still hasn't succeeded after the retries set in the retry strategy are used up, the task node is considered failed, and the Flow's failure handling strategy is triggered
In the Upstream Dependency panel, you can view the node's upstream nodes and their target tables. In edit mode, you can add and delete upstream dependencies
Flow operations management
On the Flow detail page, you can view the Flow's Operating history
Click a specific run instance to go to the Ops module and view the instance's records