Discrepancies between Facebook and Lickstats explained
Every now and then, I experience an epiphany.
Yesterday, a user reached out to report an odd discrepancy between Facebook and Lickstats for a post he was promoting. Lickstats was reporting more clicks than Facebook and Google Analytics which is theoretically impossible. Put simply, our technology cannot "create" clicks.
Last time this discrepancy was reported, I spent hours digging through our dataset and server logs but everything looked sound.
This being said, I take these reports very seriously so I decided it was time to get to the bottom of this.
After an hour of digging, everything looked fine. Almost all clicks originated from unique IPs and all had coherent user agent and geolocation data. They didn't originated from bots!
I was about to dismiss the discrepancy when I stumbled upon the following blog post titled "Is your website being hammered by Facebook and what liger has to do with it?".
A data analyst named Drazen Karacic-Soljic had experienced the same discrepancy and had launched an investigating of his own.
The result of his investigation revealed the requests originated from native Facebook apps prefetching landing pages in order to speed up load times if and when people click on links.
This is how Facebook explains prefetching:
This explains why Lickstats was reporting more clicks than Facebook. Prefetching requests are identical as "normal" clicks which explains why everything looked sound in our dataset and server logs.
Now, is it possible to filter out the prefetching requests from Lickstats you may ask? Unfortunately it's not that simple. The issue is that once a landing page has been prefetched (which results in a click), Facebook doesn't trigger a "real" click when someone clicks on the actual link. The landing page is loaded from cache.
Prefetching may cause an apparent increase in web traffic and an increase in clicks for third-party, tag-based measurement solutions. These increases may occur when marketers manually place third-party click tags in the website URL of their ads. Setting the tags up this way causes the prefetch to redirect to the third party tag, and the prefetch may then get counted as a click.
This leaves us (and all other shortlink providers) two options: filter out prefetched requests or leave them in therefore inflating total clicks.
We have chosen the latter, but thankfully, we have a secret feature to handle the discrepancy.
The Lickstats plugin (the solution)