Metadata Management Tool
1. Overview
The Metadata Management Tool is mainly used to change the type of, or delete, event properties and user properties in the AE system. Because metadata adjustments pause the ETL flow of data, we recommend using this tool only when you're sure that a type was imported incorrectly and the data is abnormal. Avoid frequent operations.
After you modify reported event properties with the secondary development tool, ask business staff to check on the Data > Management > Event Properties page whether the property connections are normal (check the Connection status column of the list, and the reported event property list on the Source Property Management page, which you open from the upper-right corner of the page. For user properties, check the Data > Management > User Properties page in the same way). This prevents dashboards, reports, tags, and other assets that already use the property from failing to calculate because of the type change.
2. Instructions
The Metadata Management Tool is available only to users of on-premises deployments. Log in as the root user to the server in the on-premises cluster where the secondary development tools are deployed (by default, the first three servers in a distributed deployment), and run su - ta
Then run ta-tool modify_meta to open the Metadata Management Tool interface.
su - ta
ta-tool modify_meta
2.1 Enter the appid of the project to process
First, enter the appid of the project to modify. You can find the project's appid on the Project Settings page of the AE system.
Enter the appid of the project whose properties you want to modify or delete >××××
2.2 Confirm the project name
After you enter the appid, the tool shows the name of the project to modify. Enter "y" to confirm or "n" to cancel
Confirm the project you will modify: Test Demo Project <y/n> y
2.3 Select the metadata operation type
Next, add the operation types to perform. You can add operations repeatedly: after you configure one operation, you can add another, until you select Finish adding. We recommend completing all modification and deletion operations at once.
Select the operation type
1. Modify event property type
2. Modify user property type
3. Delete event properties
4. Delete user properties
5. Finish adding
>
2.4 Modify event property types
2.4.1 Modify regular event properties
-
Example: Change the type of the event property argNum from number to string:
- Operation type: 1
- Event property name: argNum
- New type: string
Select the operation type
>1
The current operation modifies only reported event properties. After the modification, go to the <Data Management - Event Properties> page to check for anomalies!
Enter the name of the event property to modify>argNum
Current type: [number]. Enter the new type
number. Number type
string. String type
datetime. Datetime type
bool. Boolean type
array. List type
row. Object type
array_row. Object list type
>string
2.4.2 Modify sub-properties of objects and object groups
In the Metadata Management Tool, you can use the "parent property.sub-property" notation to point to a sub-property of an object (group) when you modify or delete it
-
Example: Change the type of the sub-property channel_id of the object (group) property argrow from number to string:
- Operation type: 1
- Event property name: argrow.channel_id
- New type: string
Select the operation type
>1
The current operation modifies only reported event properties. After the modification, go to the <Data Management - Event Properties> page to check for anomalies!
Enter the name of the event property to modify>argrow.channel_id
Current type: [number]. Enter the new type
number. Number type
string. String type
datetime. Datetime type
bool. Boolean type
array. List type
>string
2.5 Modify user property types
2.5.1 Modify regular user properties
-
Example: Change the type of the user property user_argNum from number to string:
- Operation type: 2
- User property name: user_argNum
- New type: string
Select the operation type
>2
Enter the name of the user property to modify>user_argNum
Current type: [number]. Enter the new type
number. Number type
string. String type
datetime. Datetime type
bool. Boolean type
array. List type
row. Object type
array_row. Object list type
>string
2.5.2 Modify sub-properties of objects and object groups
-
Example: Change the type of the sub-property channel_id of the object (group) property user_argrow from number to string:
- Operation type: 2
- User property name: user_argrow.channel_id
- New type: string
Select the operation type
>2
Enter the name of the user property to modify>user_argrow.channel_id
Current type: [number]. Enter the new type
number. Number type
string. String type
datetime. Datetime type
bool. Boolean type
array. List type
>string
2.6 Delete event properties
-
Example: Delete the regular event property argbool and the sub-property level of the object (group) property argrow:
- Operation type: 3
- Event property names: argbool,argrow.level
Select the operation type
>3
The current operation deletes only reported event properties. After the deletion, go to the <Data Management - Event Properties> page to check for anomalies!
Enter the names of the event properties to delete, separated by commas>argbool,argrow.level
2.7 Delete user properties
-
Example: Delete the regular user property user_argbool and the sub-property level of the object (group) property user_argrow:
- Operation type: 4
- User property names: user_argbool,user_argrow.level
Select the operation type
>4
Enter the names of the user properties to delete, separated by commas>user_argbool,user_argrow.level
2.8 Finish adding modification and deletion operations
- After you finish all configurations, enter "5" to confirm
Select the operation type
>5
The following property meta will be adjusted in project [Test Demo Project]:
Event property type changes:
argnum: number -> string
argrow.channel_id: number -> string
User property type changes:
user_argnum: number -> string
user_argrow.channel_id: number -> string
Event properties to delete:
argbool
argrow.level
User properties to delete:
user_argbool
user_argrow.level
Warning: After the operation, values of properties whose type was changed are set to NULL in imported data. Neither property type changes nor property deletions can be rolled back!
Warning: The properties you are operating on have asset dependencies, and the corresponding assets will be affected. Confirm before you proceed!
-----Event property [argrow.channel_id], impact type [regular dependency], change type [update]-----
-----Event property [argnum], impact type [regular dependency], change type [update]-----
-----User property [user_argnum], impact type [regular dependency], change type [update]-----
-----Event property [argbool], impact type [regular dependency], change type [delete]-----
-----Event property [argrow.level], impact type [regular dependency], change type [delete]-----
-----User property [user_argrow.channel_id], impact type [regular dependency], change type [update]-----
-----User property [user_argbool], impact type [regular dependency], change type [delete]-----
-----User property [user_argrow.level], impact type [regular dependency], change type [delete]-----
After the operation, values of properties whose type was changed are set to NULL. Neither modifications nor deletions can be rolled back.
2.9 Confirm the operation
- A final confirmation is requested before execution. Enter "y" to confirm or "n" to cancel
Ready to execute. Confirm (y/n)>y
Confirm again (y/n)>y
2.10 Execute the operations
After you confirm, the tool first pauses the data ETL service of the AE system. It then performs all operations in this configuration and restarts the data ETL service after all operations are complete. Data reported during this period is stored with a delay but isn't lost.
...
=== Stopping ta1 ETL service ===
=== ta1 ETL stopped successfully ===
...
Event property type modified successfully, argnum: number -> string
...
User property type modified successfully, user_argnum: number -> string
...
Event property deleted successfully, argbool
...
User property deleted successfully, user_argbool
...
----------Property modification task succeeded----------
== Waiting for the ETL component to start ==
== ta1ETL started successfully ===
----------Releasing the global update lock----------
----------Global update lock released----------
Project properties adjusted successfully
2.11 Final result
2.12 Impact on data
2.12.1 Regular properties
When you modify a regular property, the previously imported data of the property is set to null.
2.12.2 Parent properties
When you modify a parent property, the previously imported data of the parent property is set to null, including the values of all its sub-properties.
2.12.3 Sub-properties
When you modify a sub-property, the previously imported data of the sub-property is set to null. Other sub-properties under the same parent property aren't affected.
3. Notes
- While modifications and deletions are running, avoid disconnecting the session or interrupting the operation with Ctrl + C.
- Modifying and deleting properties stops the ETL flow, so avoid frequent operations that affect real-time data ingestion.
- Sub-properties can't be changed to complex structure types.