Discrepancies between Facebook and Lickstats explained

Every now and then, I experience an epiphany.

Yesterday, a user reached out to report an odd discrepancy between Facebook and Lickstats for a post he was promoting. Lickstats was reporting more clicks than Facebook and Google Analytics which is theoretically impossible. Put simply, our technology cannot "create" clicks.

Last time this discrepancy was reported, I spent hours digging through our dataset and server logs but everything looked sound.

This being said, I take these reports very seriously so I decided it was time to get to the bottom of this. 

After an hour of digging, everything looked fine. Almost all clicks originated from unique IPs and all had coherent user agent and geolocation data. They didn't originated from bots!

I was about to dismiss the discrepancy when I stumbled upon the following blog post titled "Is your website being hammered by Facebook and what liger has to do with it?".

Is your website being hammered by Facebook and what liger has to do with it?

Is your website being hammered by Facebook and what liger has to do with it?

A data analyst named Drazen Karacic-Soljic had experienced the same discrepancy and had launched an investigating of his own.

The result of his investigation revealed the requests originated from native Facebook apps prefetching landing pages in order to speed up load times if and when people click on links.

This is how Facebook explains prefetching:

For every link in mobile News Feed, Facebook attempts to predict how likely a person is to click it. If the prediction score meets certain requirements, the initial HTML file is downloaded when the link first appears on a person's screen. This content is cached locally on the person's device for a short amount of time. If the person clicks on the ad, Facebook loads the initial page from the cache. The initial page then makes regular web requests to the web server to load the remainder of the page. Facebook currently prefetches HTML for most mobile sites, but we are also experimenting with prefetching other assets such as CSS and JavaScript. Images on the website are not cached.

This explains why Lickstats was reporting more clicks than Facebook. Prefetching requests are identical as "normal" clicks which explains why everything looked sound in our dataset and server logs.

Now, is it possible to filter out the prefetching requests from Lickstats you may ask? Unfortunately it's not that simple. The issue is that once a landing page has been prefetched (which results in a click), Facebook doesn't trigger a "real" click when someone clicks on the actual link. The landing page is loaded from cache.

Prefetching may cause an apparent increase in web traffic and an increase in clicks for third-party, tag-based measurement solutions. These increases may occur when marketers manually place third-party click tags in the website URL of their ads. Setting the tags up this way causes the prefetch to redirect to the third party tag, and the prefetch may then get counted as a click.

This leaves us (and all other shortlink providers) two options: filter out prefetched requests or leave them in therefore inflating total clicks.

We have chosen the latter, but thankfully, we have a secret feature to handle the discrepancy.

The Lickstats plugin (the solution)

When Facebook prefetches a landing page, it only loads the initial HTML file, therefore it doesn't trigger JavaScript. This explains why prefetching requests are not accounted for in Google Analytics.

If you're landing people on a website you control, you can simply install our plugin which confirms clicks using JavaScript therefore mitigating the discrepancy.

Sun Knudsen