How does randomisation work?

For sure! The code is completely open-source, so you’re welcome to have a peek at the code to double check.

https://github.com/scotchfield/ab-see/blob/master/ab-see.php

Users are put into groups based on two things: their logged-in state (user id) or their IP address. If the user is logged in, A/B See will use their user id and the test id (so they’re not always in the same group) to put them in one of two buckets. If the user isn’t logged in, A/B See will use the IP address and the test id instead.

What this means is that regardless of browser, a logged-out user on a particular IP address will always be in the same bucket, and a logged-in user will always be in the same bucket.

I’d recommend watching it for a few days and seeing if it’s putting people in the right bucket. If not, please let me know! (I’ll leave this topic unresolved to be safe).

And a quick anecdote: I tried to be cute when demoing A/B testing for my team, and all five of us were put in the same group. Unlikely, but possible! ?? (The sixth person finally got added to the second group. Still made me nervous!)

Keep me posted! Thanks!

Thanks, Scott. I’ll keep an eye on it and let you know either way.

I tried with a few proxies and managed to get the China version showing up once or twice so I suspect it is working; I just happened to get Japan quite a few more times.

Great plugin. I will definitely drop you a review once everything is in order.

Awesome, and thanks! Feel free to come by and mark this as resolved when you’re satisfied that it’s working, and if it seems like something’s wrong, just let me know.

Have a great day!

Hi Scott,

I think the randomisation is working but I’d like to query some stats please.

I have set up an A/B test on my sidebar (for the ’10 great travel books’ promo on the sidebar here). In the back-end, A/B See gives me the following stats.

Group 1 conversions: 2% (46/2304)
Group 2 conversions: 1.63% (37/2268)

These clicks are unexpectedly high so I checked in Google Analytics and since the test went live, there have been 16 clicks on the destination page, not 46 + 37 as suggested by the A/B See stats.

Do you have any idea why there is a discrepancy? Any help would be much appreciated.

Thank you,

Kia

Hi Kia,
Sorry, I’m not sure why you’re seeing different numbers. I’ll try to take a closer look, but I haven’t run across this in practice.
It’s possible that the site numbers would be higher because browsers are blocking analytics or requests are timing out. Those numbers seem a lot higher though, so this probably isn’t the reason.
If you find more information, can you let me know? I’ll keep poking around on this end.
Thanks!