Xpert Ad Solutions: Comparing Reactions Results – aytm Help Center

On the Results page, the Xpert Report is presented as the default report. The Original - as fielded report listed below contains the entire survey, including any additional questions you added to the Xpert Ad test.

1. Objective and Methodology

Click on the Objective and Methodology tabs to expand the info, which will be included in any Powerpoint exports, so you can share the details of the ad test with your colleagues.

One of the features of Xpert Reports is endorsement. Note that, since this test asks respondents to directly compare ads, the methodology summary reflects that this information is used to give a recommendation about ad performance. An attribute is considered “endorsed” when the percentage of respondents who selected the Top 2 Boxes exceeds a given threshold. By default, this threshold is set at 70%. The next two drop-downs indicate confidence level for the concept lab testing, and are automatically set at 95% confidence for statistical significance, and 90% confidence for marginal significance.

In the Methodology tab, click the Threshold drop-down to select a threshold of Top Two respondents that meets your criteria for endorsement.
Click the respective drop-downs for statistical significance and marginal significance to adjust these to your alpha standards.

2. Recommended Advertisement

Recommended Advertisement summarizes results, recommending which of your proposed ads achieved the highest overall mean attitude score, which is endorsed by included attributes, and a preference score derived from the side-by-side comparison results. If the results indicate that “none” were endorsed, you can lower your threshold in the Methodology tab, and the results will auto-update. Based on those pieces of information, an overall recommendation will be made.

3. Table 1: Overall Attitude Score

In Table 1: overall attitude score, the table lists each ad image or video, each attribute’s mean score on a scale of -50 to +50, the standard deviation, and, if applicable, which ads are statistically significantly different. If an advertisement is significantly different at the 95% confidence level, a capital letter indicating which other advertisement(s) it is different from will appear in the significance column. If they are only marginally different, at the 90% confidence level, a lower case letter will appear. If you have adjusted confidence levels to a different alpha standard, the table will automatically update.

4. Table 2: Endorsed Characteristics

Table 2 Endorsed Characteristics displays the percent of people who gave a Top 2 Box response to each attribute. When an ad exceeds the set threshold for a certain attribute, the cell will be highlighted, and considered “endorsed” for that attribute.

5. Table 3: Preference Score

Table 3: preference score gives a narrative summary of results, and in the table, shows the mean, standard deviation, and statistical significance. As in Table 1, a capital letter will show if the option received 95% significance, and a lowercase letter will show if the option received 90% significance.

6. Overall Attitude at Top 2 Box

The next section, Overall Attitude at Top 2 box, shows a chart for the percentage of respondents who gave a score of 25 or higher to each proposed ad. Hover over the chart to see specific metrics, which are also displayed line-by-line below the chart.

7. % of respondents for whom the Advertisement is their First Choice

Based on the results of the side-by-side comparison, XQ8 shows a chart of the percentage of respondents who picked each advertisement as their first choice. Hover over the bars or view the line-by-line data to see actual percentages and number of responses. If you would like to see the actual side-by-side widget, click on the Original - as fielded report, and find the question there.

8. Concept Lab

The Concept Lab section allows you to compare the Top 2 Box percent of each concept in a column view with all attribute scores, including your custom attributes.

Click the Analysis drop-down to view as a Sigtest or Row averages.
Click the Show drop-down to display the Top 3 performers, the Top 25% of performers, All ads, or choose none to clear all, and add them manually using the + button.
Click the Rank by: drop-down and select an attribute, or click on the attribute in the lefthand column to sort performance by any of the attributes. The ads rearrange places, left to right, and the ranking attribute appears in bold.

9. Advertisement Details

For each concept populated in XQ9, drill-down details are displayed in the next tab. You can scroll down to get to those details, or hover over the thumbnail, and click Scroll to details. This chart shows a bar graph of the Top 2 box percentages for all included attributes of a concept. Hover over the bars to see statistical significance, and scroll down to see line-by-line results.

Below the included attributes bar graph is a Top 2 box bar graph and line-by-line data for your custom attributes.

Below any custom attribute data, are results for the two included Open-ended questions; an overall summary of the most common words used, and a word cloud for the question “what is the overall message of the advertisement.” Below this word cloud, there are side-by-side word clouds that show the answers to What did you like about the ad and What did you dislike about the ad, which are shown only to respondents who indicated preference one way or the other.

If you would like to see drill-down details for each concept, make sure that all concepts are added in the Concept Lab section above.

10. Comparing Users and Non-users

If you selected the user/non-user comparison option, those results are displayed in XQ10 and following. This gives a summary report, showing how many buyers and non-buyers responded. Keep in mind that significance testing can only be conducted if there are at least 30 in each group. However, even with fewer than 30, XQ10 will still show the overall attitude score and then number of respondents for each ad concept.

If there are not enough respondents in each group, XQ11 will warn that we cannot test for significant differences in overall attitude between buyers and non-buyers. Concept labs will appear for each group that has at least 30 respondents, so you can compare how ads performed within the respective group.

To further explore the results for Xpert Ad tests, go to the Original as fielded report, and utilize Virtual Questions and Correlation finders, or export raw data files.

Articles in this section