Using FairlyCertain

Installing FairlyCertain is quite straightforward. I built it so that it would simply drop into an existing project with the least amount of disruption.

Before following these instructions, you should download the library and sample project.

Copying files

You'll need to copy some of the files from the source distribution into your project and into places that the project can find them.

• Drop the /ABTesting directory straight into your project, either right at the root or into a /Helpers folder if you have one. Alternately, if you have a separate project for reusable utility code, you can drop it there.
• (optional) Copy the ABDashboard.aspx file from the sample project into your project, preferably in the same directory you keep the rest of your Admin dashboards and tools. Be sure to modify it so that only Admins can see it.
• (recommended) Configure whatever script that deploys your site so that it ignores the file "/tests.ab". That's where your stats are stored, so if you wipe it you wipe them.

Starting a new A/B Test

There are several ways to set up a new test, depending on where you're putting it.

Using the <ab:Test> UserControl

FairlyCertain comes bundled with a simple UserControl that you can use on .aspx, .ascx, and .master pages. Using it works just like any other custom UserControl. You simply Register it at the top of the page, then place it wherever you like.

The code below is what we're using to test variations of the sidebar text on this site:

<%@ Register TagPrefix="ab" Namespace="ExpatSoftware.Helpers.ABTesting.Controls" Assembly="Web" %> <ab:Test TestName="sidebar_intro_text" runat=server> <ab:alternative Name="learn_how" runat="server"> It lets you quickly test changes to your website to learn how real users respond to them. </ab:alternative> <ab:alternative Name="sell_more_stuff" runat="server"> It lets you test website changes against each other so that you'll sell more stuff and make more money. </ab:alternative> </ab:Test>

Calling FairlyCertain.Test() directly

The other way to set up an AB test is to simply call FairlyCertain.Test().

For example, this is all the code you need to start a test called "red_blue" that picks between those two colors: string color = FairlyCertain.Test("red_blue", "red", "blue"); In ASP.NET MVC (or if you just don't like UserControls), you can do this: <% if (FairlyCertain.Test("bold_test", "bold", "standard") == "bold") {%> AB Testing is <b>freaking awesome</b>. <% } else { %> AB Testing is freaking awesome. <% } %> ... or simply this: <% if (FairlyCertain.Test("show_disclaimer")){%> (possible side effects may involve vomiting and hair loss) <% } %> Regardless of how you do it, I hope you'll agree that it's not a lot of code, and it's pretty clean.

Scoring Conversions

Once your user succeeds in doing whatever you wanted them to do, you can record the conversion by simply calling FairlyCertain.Score(). As in: FairlyCertain.Score("sidebar_intro_text"); or: FairlyCertain.Score("bold_header","yellow_background","30_day_trial");

Calculating Statistics

I've included a simple A/B Dashboard page that displays a pretty(ish) report of all your active tests, with conversion information, percentages, and a bit of text explaining what the results mean.

Assuming you're lazy, all you need to do is copy that page (ABDashboard.aspx and it's codebehinds) into your project, touch it up to make sure it's secure, and then link to it from your Admin screen.

If you want more detail or better presentation, the FairlyLocal and ABTest objects expose plenty of statistics methods to help you out. And of course you can pull out any raw data that you might need straight from the data structure.