Integrating your Experiment

Now we will cover implementing an A/B test in your application. We will continue with the example scenario used in Defining your Experiment.

Example Scenario:
You are developing an application with a button allowing users to activate a newly released feature. This button currently has the label "Get Started Now!" with a green background color.

You would like to increase the number of users who click on this button.

You hypothesize that the label "GET STARTED NOW!" with a red background color may achieve this, and wish to run an A/B test to determine if this variation is, in fact, a better choice.

Retrieving an Experiment

The first step in using an experiment in your application is to retrieve it from the Kii Cloud. This will get the A/B test definition and status of the experiment (e.g., the experiment is ongoing or has been completed).

Sample code for Android:

// Get the experiment synchronously.
KiiExperiment experiment = null;
try {
  experiment = KiiExperiment.getByID("7a9d430f-fef6-424d-a521-7e07318650fa");
} catch (AppException e) {
  // Handle the error.
} catch (IOException e) {
  // Handle the error.
}

Sample code for iOS:

Swift 3:

// Get the experiment synchronously.
let experiment : KiiExperiment

do{
  experiment = try KiiExperiment.getSynchronous("7a9d430f-fef6-424d-a521-7e07318650fa")
}catch let error as NSError {
  // Handle the error.
  return
}

Objective-C:

// Get the experiment synchronously.
KiiExperiment* experiment = nil;
NSError* error = nil;
experiment = [KiiExperiment getExperimentSynchronous:@"7a9d430f-fef6-424d-a521-7e07318650fa"
                                           withError:&error];
if(error != nil){
  // Handle the error.
}

In all cases, the getByName/getExperimentSynchronous/GetByID methods are called with the ID of the target experiment. As explained in Defining your Experiment, you can find this ID in the developer portal.

If the application fails to fetch the experiment for some reason (e.g the device is offline), an exception will be thrown. For more information, please see the Javadoc, appledoc, or Monodoc.

Apply a Variation

Now that your experiment has been retrieved, we can get the chosen variation and variable values (based on the experiment definition).

Sample code for Android:

// Assume these are the variables for variation 'A'.
String buttonColor = "green";
String buttonLabel = "Get Started Now!";

// Initialize a variation. This sample code fallbacks to variation 'A'.
Variation va = experiment.getVariationByName("A");
try {
  va = experiment.getAppliedVariation();
} catch (ExperimentNotAppliedException e) {
  // Failed to apply a variation.
  // Execute the getReason() method for the cause of this error.
  // This sample code fallbacks to variation 'A' if it fails to randomly apply a variation.
}

JSONObject test = va.getVariableSet();
try {
  buttonColor = test.getString("buttonColor");
  buttonLabel = test.getString("buttonLabel");
  // Apply the color and label to the button.
} catch (JSONException e) {
  // This error occurs when the configuration in the developer portal and the code differ.
  // Apply the variables for variation 'A' that are declared above.
}

Sample code for iOS:

Swift 3:

let variation : KiiVariation
do{
  variation = try experiment.appliedVariation()

}catch{
  // Failed to apply a variation.
  // Get the error.code property for the cause of this error.
  // This sample code fallbacks to variation 'A' if it fails to randomly apply a variation.
  variation = experiment.variation(byName: "A")!
}

let variableSet = variation.variableDictionary!

// Get the variables for variation 'A'.
let buttonColor = variableSet["buttonColor"] as! String
let buttonLabel = variableSet["buttonLabel"] as! String
// Apply the color and label to the button.

Objective-C:

NSError *error;
KiiVariation *variation = [experiment appliedVariationWithError:&error];
if (error != nil) {
  // Failed to apply a variation.
  // Get the error.code property for the cause of this error.
  // This sample code fallbacks to variation 'A' if it fails to randomly apply a variation.
  variation = [experiment variationByName:@"A"];
}

NSDictionary *variableSet = variation.variableDictionary;

// Get the variables for variation 'A'.
NSString *buttonColor = variableSet[@"buttonColor"];
NSString *buttonLabel = variableSet[@"buttonLabel"];
// Apply the color and label to the button.

Here is a brief explanation of the code above:

  1. Get the variation by calling the getAppliedVariation/appliedVariationWithError/GetAppliedVariation method.

  2. Get the variable values by calling getVariableSet/variableDictionary/VariableSet method.

Note that getting the variation might fail. Some possible reasons for this failure include:

We advise you to handle these situations in your code, which can be achieved in most cases by using the default variable values as illustrated in the code samples. (You can also get the failure reason by calling the getReason method (Android) or by checking the error code (iOS). Please check the Javadoc/appledoc/Monodoc for details).

Sampler variation without user login

The default variation sampler uses KiiUser as random seed generator. Thus, it requires user login.

To apply sampler without requiring user login, you can use time-based variation sampler. The following sample code will guide you to implement variation without a necessity for user login.

Sample code for Android:

// Assume these are the variables for variation 'A'.
String buttonColor = "green";
String buttonLabel = "Get Started Now!";

// Define a sampler based on the execution timestamp.
VariationSampler randomSampler = new RandomVariationSampler();

// Initialize a variation. This sample code fallbacks to variation 'A'.
Variation fallback = experiment.getVariationByName("A");
Variation va = experiment.getAppliedVariation(fallback, randomSampler);

JSONObject test = va.getVariableSet();
try {
  buttonColor = test.getString("buttonColor");
  buttonLabel = test.getString("buttonLabel");
  // Apply the color and label to the button.
} catch (JSONException e) {
  // This error occurs when the configuration in the developer portal and the code differ.
  // Apply the variables for variation 'A' that are declared above.
}

Sample code for iOS:

Swift 3:

// Assume these are the variables for variation 'A'.
var buttonColor = "green";
var buttonLabel = "Get Started Now!";

// Define a sampler based on the execution timestamp.
let randomSampler =  KiiRandomVariationSampler()

let fallback = experiment.variation(byName: "A")
let variation = experiment.appliedVariation(with: randomSampler, fallback: fallback)!

let variableSet = variation.variableDictionary!

// Get the variables for variation 'A'.
buttonColor = variableSet["buttonColor"] as! String
buttonLabel = variableSet["buttonLabel"] as! String
// Apply the color and label to the button.

Objective-C:

NSError *error;
// Define a sampler based on the execution timestamp.
id<KiiVariationSampler> *randomSampler =  [[KiiRandomVariationSampler alloc] init];
// Initialize a variation. This sample code fallbacks to variation 'A'.
KiiVariation *fallback = [experiment variationByName:@"A"];
KiiVariation *variation = [experiment appliedVariationWithSampler:randomSampler
                                                         fallback:fallback];

NSDictionary *variableSet = variation.variableDictionary;

// Get the variables for variation 'A'.
NSString *buttonColor = variableSet[@"buttonColor"];
NSString *buttonLabel = variableSet[@"buttonLabel"];
// Apply the color and label to the button.

Sending Events

The final step is to send the view/conversion events from your application. In our example scenario, we will send the following two events (see Defining your Experiment to see how we defined these events in the developer portal):

  • The event eventViewed when the target button is displayed to the user.
  • The event eventClicked when the target button is clicked.

Sample code for Android:

// The button is displayed and the "eventViewed" event is triggered.
KiiEvent viewEvent = va.eventForConversion(getApplicationContext(),
                                           "eventViewed");

// The button is clicked and the "eventClicked" event is triggered.
KiiEvent clickedEvent = va.eventForConversion(getApplicationContext(),
                                              "eventClicked");
try {
  // This sample code does not consider the user operation context.
  // In a real app, send it to the handler of the Activity and View classes.
  viewEvent.push();
  clickedEvent.push();
} catch (IOException e) {
  // This error occurs when the app storage is not accessible.
  // This is generally a rare exception and handling it is out of the scope of this sample code.
}

Sample code for iOS:

Swift 3:

// The button is displayed and the "eventViewed" event is triggered.
let viewEvent = variation.eventDictionaryForConversion(withName: "eventViewed")

// The button is clicked and the "eventClicked" event is triggered.
let clickEvent = variation.eventDictionaryForConversion(withName: "eventClicked")

// This sample code does not consider the user operation context.
// In a real app, send it to the handler of the controller class.
KiiAnalytics.trackEvent(experiment.experimentID, withExtras: viewEvent)
KiiAnalytics.trackEvent(experiment.experimentID, withExtras: clickEvent)

Objective-C:

// The button is displayed and the "eventViewed" event is triggered.
NSDictionary *viewEvent = [variation eventDictionaryForConversionWithName:@"eventViewed"];

// The button is clicked and the "eventClicked" event is triggered.
NSDictionary *clickEvent = [variation eventDictionaryForConversionWithName:@"eventClicked"];

// This sample code does not consider the user operation context.
// In a real app, send it to the handler of the controller class.
[KiiAnalytics trackEvent:experiment.experimentID
              withExtras:viewEvent];
[KiiAnalytics trackEvent:experiment.experimentID
              withExtras:clickEvent];

Here is a brief explanation of the code:

  1. Call the eventForConversion/eventDictionaryForConversionWithName/EventForConversion method with the event name to record the event.
  2. Call the push/trackEvent/Upload method to send the recorded events to Kii Cloud.

Calling the push/trackEvent/Upload method will cache the event data locally. It will be automatically uploaded to Kii Cloud at an optimal time.


The experiment and application are now set up. We are ready to begin the experiment: Executing your Experiment