Experiments

You can create Experiments to test certain configuration changes on a selective range of Players within your game. For example, you could experiment with the costs of Virtual Goods for a small number of Players on your game and for a specified period of time. You might increase or decrease the costs of some Virtual Goods to see how Players would respond to these different costings for Virtual Goods. By reviewing the outcomes in terms of player behavior of such experimental adjustments in Virtual Goods costs, you'll be very well-placed to fine-tune your game configuration to make Players more responsive to and more engaged with your game.

What Parts of my Game Configuration can I Experiment with?

You can create and configure Experiments to differentiate your players' game experience of the following:

Creating Experiments

To create an Experiment, click Experiments in the main navigation bar. The Experiments page opens:

From here you can create and manage your Experiments:

Adding and Configuring an Experiment

To create and configure an Upcoming Experiment, you must work through a series of four steps:

1. On the Upcoming Experiments panel, click to Add a new Experiment. You're taken to Step 1 - Experiment Period:

Use this first step to enter:

2. Click Next. You're taken to Step 2 - Variants:

Use this second step to select what you want to vary for the purpose of the Experiment:

This means that any players who become participants in the Experiment will be charged currency costs that vary from the regular costs when they purchase either a gold or silver coin Virtual Good.

3. Click Next. You're taken to Step 3 - Player Pool and Variant Configuration:

You can use this step to configure several aspects of the Experiment:

Player Inclusion! Only players registered at the time the Experiment starts running are included in the overall player pool - any players that register after the Experiment starts are not included.

These combined settings are designed to ensure that when your Experiment runs, the set of players who become participants in the Experiment is a randomly chosen set. This random selection of participants means that any behavioral changes observed in the players subjected to the Experiment with respect to the Variants - here Virtual Goods whose currency values have been increased or decreased - are therefore less likely to be biased and unrepresentative. This means, in turn, that any decisions you make for actual changes you go on to make to your game configuration and which are based on the observed results of the Experiment are much more likely to be predictive of the general behavior of all game players in reaction to these actual changes.

Note: We make a best effort to calculate the percentage ranges, however for smaller sample sets of players you might not be returned the exact number of players as anticipated.

Using Calculated Variant Values? Instead of entering an absolute value as a variant value for a Currency award, you can use a calculated value. For example, if the original Currency value is 100, you can use +20 or -20 to make the variant value 120 or 80. Secondly, you can enter a calculated value by percentage. For example, if the original Currency value is 100, you can enter +15% or -15% to make the variant value for the experiment 115 or 85. If you use calculated variant values for named Currencies in your Experiment, the calculation is represented as: +20 x Currency Name - this means 20 will be added to the normal currency value.

Compounding Calculated Values! Note that if you intend to run Experiments against configurables you have segmented using calculated values, you must be careful when also using calculated values for your experimental variant values - see Segmentation and Running Experiments on the Segments page for details on this issue.

Browser Zoom! Please maintain your browser at normal zoom level when you are adding your percentile ranges.

4. Click Next. You are taken to Step 4 - Measurements:

Your measurements for the Experiment could be based on a request that you expect the participant players to use. If they've used it during the Experiment, we can see the percentage of players that were in the Experiment and how many from each Variant of the Experiment submitted the request:

5. Click to Save and Close the Experiment. The new Experiment is added to the Upcoming Experiments panel.

This shows the Start and End date you have configured for the Experiment and under Actions there are several options:

Running Experiments

When the start date for an Upcoming Experiment has passed or you have started the Experiment manually, it moves to Running Experiments:

Under Actions there are several options:

Completed Experiments

When a Running Experiment has reached its end date or you have stopped it manually, it moves to Completed Experiments:

Under Actions there are several options:

Viewing Experiment Results

Once your Experiment has completed, you can check the results of your Experiment and the impact it has had on the pool of Players who were affected by this Experiment:

1. In the Completed Experiments panel click to view the results of an Experiment:

The Results page opens:

2. Scroll down the page to review and analyze the results, which are shown in two different panels:

Results Delay! If you stop an Experiment and try to view results straightaway, you will not get the finalized results. This is because there is a slight delay of approximately ten minutes before the data for the activity being measured in the Experiment becomes available to build the results charts.

Publishing Experiments

Experiments can always be run in your game's Preview stage. If you want to run your Experiment in your game's Live stage, you must publish your Experiment:

1. When you have an Experiment ready for the Live stage, in the Upcoming Experiments panel, click the publish icon:

A confirmation dialog appears:

2. Click Ok. The Experiment is published to the Live game stage and shows in green on the Upcoming Experiments panel:

Did this page help you? Please enter your feedback below. For questions about using this part of the platform, please contact support here