Fascinating article from Jakob Nielsen highlighting a real problem for measuring the effect of marketing on internet usage.

An experiment on whether users were more or less likely to visit Yahoo having seen a promotional video showed the following: the thick blue line shows a 144% increase in likelihood of visits to Yahoo on the day of the video and some additional likelihood the following day. This looks like a direct impact of the promotional video on visits to Yahoo. Score one for internet advertising.

However, two major things aren’t right here.

1. The blue line for Yahoo visits lifts the day before the viewing of the video: time travel?
2. The thin orange line shows people who were shown an unrelated video rather than the Yahoo promotional one. The user behaviour is identical.

Internet activity bias explains what was going on:

Users in both the stimulus and control groups were simply more active on the Internet the day they watched the videos.

This phenomenon is called activity bias:some days, people do a lot online;other days, they do very little.

This is terrible news for people who want to measure the effect of advertising on internet usage. If you see a spike in usage, you might be tempted to claim the efficacy of a recent campaign. Most likely, though, is that people are just more active that day.

The researchers carried out additional testing which showed that even with specific ads turned off, people would still end up shopping at the sites covered by those ads. Why? Ad homeopathy – the memory of where an ad might have been? No, because the user was on a busier internet activity day and wanted to buy. They would get there with or without the ads.

Humour: the german article title is Internet Activity Bias verursacht klumpiges Nutzerverhalten. I love it.

Original article: Internet Activity Bias Causes Lumpy User Behavior.

Link: http://www.nngroup.com/articles/internet-activity-bias/