Defining your Experiment

We will cover defining an experiment through the use an example scenario.

Example Scenario:
You are developing an application with a button allowing users to activate a newly released feature. This button currently has the label "Get Started Now!" with a green background color. You would like to increase the number of users who click on this button.

You hypothesize that the label "GET STARTED NOW!" with a red background color may achieve this, and wish to run an A/B test to determine if this variation is, in fact, a better choice.

Creating a New Experiment

The first step is to create an experiment that defines this A/B testing scenario.

  1. Log in to the developer portal and select your application.
  2. Click the "A/B tests" icon. This will bring up the A/B Testing console.

  3. Click "Create a Test" to start defining your experiment.

    Enter the following information, referring to the screenshot above.

    • Add a Test name and Description for your experiment.
    • Define Events to track actions in the experiment.
      You will need to define two types of events:
      • View Event: The event triggered when the A/B test begins (the view event).

        In our example scenario, we assume that the event eventViewed is triggered when the application displays the button to the user.

      • Conversion: The target event to be measured (the conversion event).

        In our example scenario, we assume that the event eventClicked is triggered when the user clicks the button. We also specify "Button clicked" for the description of this conversion event.

    • Define the Variation A (Control) and B.
      • Set the Distribution ratio between Variation A (Control) and B using the slider.
      • Define Variables for each variation by setting their Names and Values.

        In our example scenario, we assume that the label and color of the target button are controlled with the variables buttonLabel and buttonColor. The corresponding variable values are set for each variation.

    Click the "Save" button when done.

  4. Confirm the experiment summary is listed on the console.

  5. You can check the experiment ID by clicking the "ID" icon. You will use this ID later when we implement the A/B test in the application.

  6. To review your experiment, click the experiment summary.

    As you can see, the experiment ID will be automatically inserted in your experiment definition.

That's it! Now the A/B test has been defined as a new experiment.


Let us now see how we implement the A/B test in the client application: Integrating your Experiment