Config effect analysis
Starting from V5.0, you can use event tracking and metric settings to analyze the effect of in-app content delivered through Config Center, and compare the effect data of any two config strategies.
Tracking instructions
Config analysis reports rely on collecting tracking data for the delivered config content. When you deliver configs with AE Config Center, the delivered content carries the AE receipt field #ops_receipt_properties. Taking webhook delivery as an example, the delivery format is as follows:
[{
"opType": "${opTypeValue}",
"configInfo": {
"configId": "${configIdValue}",
"templateId": "${templateIdValue}",
"strategyId": "${strategyIdValue}"
},
"targetType": "${targetTypeValue}",
"targetEnv": ${targetEnvValue},
"targetAudience": ${targetAudienceValue},
"configParams": ${configParamsValue},
"#ops_receipt_properties": ${#ops_receipt_propertiesValue}, //This field carries the strategy ID, template ID, config item ID, and strategy version number.
"isEndPush": ${isEndPushValue}
}]
You need to report the AE receipt field #ops_receipt_properties in the delivered content as a pass-through property of the tracking events related to your config business, and bind the tracking events (or the analysis metrics created from them) to the analysis settings of the config item. After this is set, when you view a strategy report, the system automatically filters by strategy ID (or template ID or config item ID) based on the pass-through property values in the tracking events, and generates effect analysis reports for different strategies (or templates or config items).
Analysis settings
Under each config item, you can configure the effect metrics to analyze for that config item. AE provides two ways to configure metrics:
- Predefined events
When you bind business events to the preset event slots, the system automatically generates business event metrics:
- Exposure event: exposed users, exposures, exposure rate (exposed users / released users)
- Click event: clicking users, click times, click-through rate (clicking users / exposed users)
- Participation event: participating users, participate times, participate rate (participating users / clicking users)
- Custom analysis metrics
You can specify metrics that have already been created in Data > Management > Metrics as config analysis metrics. For example, you can associate gift pack payment metrics with gift pack configs, or bind campaign analysis metrics to campaign configs.
By default, when you view a data report, custom metrics are calculated automatically based on what you are viewing. For example, if you use "Average revenue per campaign participant" as a custom metric, when you view a specific campaign instance (a specific config strategy), the system filters the data of that campaign instance directly based on the config pass-through parameters carried in the tracking events. This enables effect analysis at the config item, template, and strategy levels.
Depending on your analysis goal, you can also click View Rules to specify whether the events used by the metric are filtered by config strategy. Example scenarios:
| Example metric | Filter type | Calculation logic |
|---|---|---|
| Average revenue per campaign participant (campaign revenue / campaign participants) | Filter all | Filtered automatically based on what you view. |
| Share of active users who participate in the campaign (campaign participants / total active users) | Filter specified events | Only participating users are filtered based on what you view. |
| Overall payment rate (shows the overall payment rate in the data report as well) | No filter | Not filtered based on what you view. |
Data reports
Click Data Analysis in the left navigation of Config Center to open the config data report, or click Analyze next to a specific config strategy to view the data report of that strategy.
Analysis operations
- Group or filter by template
Click Template in the upper-left corner of the data analysis report to group the data results by template. After you switch to the template view, you can select the templates you are interested in from the template drop-down list to view their data.
- Group or filter by strategy
Click Strategy in the upper-left corner of the data analysis report to group the data results by strategy. After you switch to the strategy view, you can select the strategies you are interested in from the strategy drop-down list to view their data.
Strategy data comparison
You can select any two strategies that have gone online and compare their metric data.
Metric descriptions
| Metric | Description |
|---|---|
| Exposure rate | Shown only when all strategies within the statistical scope deliver configs to an Audience. (Not counted for system-level config delivery) |
| Click-through rate | Shown only when the Exposure and Click events are configured |
| Participate rate | Shown only when the Click and Participate events are configured |