To create and send a Holdout Experiment, follow the steps below:
Step 1
1.1 Navigate to Communications in the portal
1.2 Under the email section, select scheduled
1.3 Select the Experiments Tab
1.4 Click create new email
Step 2
2.1 Create a name for the email
2.2 Select Holdout Experiment
2.3 Choose the Key Performance Indicator, average order value, total purchases, or spend per customer
2.4 (optional) track the email with Google Analytics
2.5 Click Compose Email
Step 3
3.1 Choose the sender
3.2 Create a subject for the email
3.3 (optional) start design from a saved template, select template and load
3.4 Send a Test Email, see here for more details
3.5 Open Asset Manager if needed. This will open a new window and provide access to all images you have hosted in our Asset Manager. We recommend storing your images in our platform to ensure all images are populated when sending an email
3.6 Open Merge Variables if needed. This will open a new window and provide a list of all available merge variables to be used in your email communication
3.7 Compose your email
3.8 Select Add Recipients
Step 4
4.1 Select the subscriber list
4.2 Option to filter list by segment
4.3 Choose your confidence level – degree of certainty you’d like to have in the results (Higher confidence will require more recipients and more time)
4.4 (optional) Adjust the baseline response rate and the minimum detectable effect
4.5 Marketer does not pick sample size for experiment – but is given insight into how many people will be getting the test. You will be able to see the total number of people who will receive the email and the number of people who will be held out of the experiment
4.6 Select Schedule
Step 5
5.1 Decide if you would like the message to be sent right away
5.2 Decide if you would like to schedule the message to be sent for a certain date and time, which you can select
5.3 Select the Experiment Duration
5.4 Click Review to move on to the next step
Step 6
6.1 Review your message – confirm all information and select Submit
