AE system data export
This is a value-added feature of the DataOps Platform. To enable it, contact ThinkingAI operations to turn on the system-level switch.
System table usage
The AE system accumulates rich data assets, including:
- System feature tables: System tables maintained by features such as system analysis, data management, and tags and cohorts
- Dimension tables: Dimension tables that users upload and create themselves
- Operation behavior tables: Operation process data such as user outreach and push records
With the data integration capabilities of the DataOps Platform, you can efficiently export these AE system data assets to your own operations analysis systems, helping your enterprise dig deeper into the value of its data and drive business growth.
System table authorization
A space can set authorization separately for each project. There are three levels:
- Allow supply and use
- Allow use
- No authorization
"Supply": Assets processed in the DataOps workspace can be supplied to the selected project for use
"Use": The DataOps workspace can use assets in the selected project for processing
Therefore, to use the System table data recorded in a Project within a DataOps Platform space, the DataOps workspace first needs the Use permission for that project.
System table types
- Currently, the system tables that can be authorized for external use only include system tables in the Hive warehouse, not system tables recorded in Kudu or MySQL.
- You only have Query permission on system tables, without DML or DDL permissions.
- The system tables that can be authorized are managed with an allowlist. The system table types currently available are listed below. For other needs, contact your account manager.
| AE table type | Data catalog | Database name | Table name pattern | Table description (note: same as the dictionary area of SQL IDE) | Available engines | Remarks |
|---|---|---|---|---|---|---|
| User Snapshot Fact Table | hive | ta | ta.user_day_serial_<project ID> | User Snapshot Fact Table | Trino, StarRocks | |
| Tag and Cohort Table | hive | ta | ta.user_result_cluster_<project ID> | Tag and Cohort Table | Trino, StarRocks |
Export system tables
Go to DataOps Platform → Integration → Offline Sync and click + Integration plan to open the integration plan creation page.
In the data source settings, select Built-in Warehouse - AE System (shown as TE System in the UI), then select the db and table you want to export.
Click Data Preview to view sample data from the system table
Next, configure the target data source and the field mappings to finish configuring the plan for exporting system table data.
For how to mount an integration plan on a Flow to run it, see Mount on Flow in the previous section.