Data tables
Data tables are one of the data ingestion methods the system provides; the other is SDK integration. Customers can import their own data into data tables for analysis.
Currently, data tables can be used as dimension tables for event properties and user properties, and you can update the data in a data table by importing table files.
1. Create a data table
Click Create Data Table and choose File upload or Dynamic SQL Table to create a data table. If you choose File upload, import a table file to create a new data table (see how to save CSV files). Note that Excel files can't exceed 100M and CSV files can't exceed 500M. If your data file exceeds these limits, you can import it in multiple batches through incremental updates after the table is created.
The column names of the new data table are the same as the column names (first row) of the uploaded table file, and you also need to specify the data type of each field. Note that the data type of the table's primary key can't be changed after creation, and it determines which properties the table can be associated with, as shown in the following table:
| Property data type | Adding a dimension table |
|---|---|
| String, Number, Boolean | Supported. The table's primary key must have the same data type as the property. |
| Time | Supported. The table's primary key must be String and match the specified time format. |
| List | Supported. The table's primary key must be String. |
| Row, Array Row | Dimension table properties not supported. Child properties of a Row can have dimension tables. |
On the preview page, check whether the data in each column is parsed correctly. You can get the rows that failed to parse by downloading the error details.
The data in the table file is automatically inserted into the new table. If different rows in the file have the same value in the first column, they are deduplicated before being inserted into the data table, keeping the row with the smallest row number among them. (Tips: Because the schema treats the first column of the uploaded table as the primary key by default and primary key values can't be duplicated, deduplication happens automatically)
2. Update data
Incremental Update is a lightweight update method: you only need to import the small amount of data that has changed. If a primary key value in the imported data already exists in the data table, that row is updated; if it doesn't exist, a new row is inserted. This method doesn't affect data whose primary key values don't appear in the imported table file.
Because the size of a single uploaded file is limited (Excel files can't exceed 100M and CSV files can't exceed 500M), if you need to update a large amount of data, you can import it in multiple batches.
For an incremental update, the uploaded table must have exactly the same structure as the current data table, including column names and column order.
When a large range of data changes or you need to delete some data, Replacement Update is recommended. This method first clears the data in the original data table and then writes all the data of the imported table into it. The table file requirements are the same as for incremental updates.
You can also update data tables through the Data Table API . If you're using a dimension table created before v3.8, you can update its data through the Dimension Table API.
3. Manage data tables
On the data table page, you can perform management actions on created data tables, such as updating the table structure, deleting a table, and restoring a version of the table and its data.
To change the display names of table fields, use Incremental Update. To add or remove columns, or change the data type of a column (the primary key field doesn't support data type changes), use Replacement Update.
Table structure changes affect the assets that use this data table and take effect only after you Publish them. The publish status column of the list also shows when a data table has unpublished updates. When you publish, you need to confirm the table structure that goes live and its impact.
If you need to roll back after changing a data table's data or structure, you can restore a version from the data table's version history. Note that the restored content includes the data table structure and the data in the table, but does not contain the associations between the data table and properties.
A data table can't be recovered after it's deleted. If the data table has been added as a dimension table, deleting it also deletes the dimension table properties created from it.
You can also manage data tables through the Data Table API .
4. Use data tables
On the data table page, you can directly add a data table as the dimension table of one or more properties. For details, see Associate data tables with properties. You can also add a data table as a dimension table on the User Properties and Event Properties pages.
5. Feature permissions
The role permissions involved are as follows:
| Feature | Permission | Details |
|---|---|---|
| Metadata | View | View the lists of events, event properties, and user properties; preview data tables |
| Metadata | View | Export dimension tables of properties; download data table content and historical content |
| Metadata | Edit | Add and edit file-upload data tables; restore historical versions of file-upload data tables |
| Metadata | Edit | Add and edit data tables created with SQL statements |