Unethical: testing with cookie wall button placement

I'm guilty. Sometimes I've done tests that are non ethical or not allowed any longer due to (EU in my case) data regulations. 

Luckily there are some rules now and ethical considerations are important for any CRO / Experimentation professional. 

I could pretend to be an angel — but rather I'd like to share my learnings with you so you don't have to repeat these experiments.

Cookie wall CTA placement

Why this is not allowed: you're only allowed to use A/B test data after receiving permission of using cookies and data. Not before. 

How to do it right? Count accepts per treatment. Assuming your randomization works and users will see the modal once. You don't track users that you reject, so you might get: Control 12467 accepts vs 13922 accepts. Then you can use a chi-square test (basically the same as an SRM check) to see if the difference in total accepts for each variation is significantly different from what would be expected by chance - thanks to Lucas Vermeer to help out on this topic. 
 

Cookie walls: improving CTA placement

On desktop: place the accept button on the right bottom of the cookie wall. This takes away one extra step in reading and scanning the popup. 

On mobile: place the accept button on the bottom of the cookie wall. Why easiest place to reach with thumb? And same as above… the last thing you read is the most important one. 

Read the discussion on this topic on Linkedin.