Normally, a newsletter’s click-through rate (CTR) is an indication of its quality.
But that rule of thumb might be in jeopardy.
In the last few months, Paved has recorded a rapid increase in automated clicks. These ‘fake’ clicks from email security bots, firewalls, and spam filters.
Most people usually assume email is free of the bot-related problems that plague other advertising platforms. An email address –more often than not — has a real person behind it.
But bot traffic in email is a growing problem.
For most publishers, around 20% of clicks come from automated bots. In extreme cases, publishers with a large number of business emails are seeing up to 90% bot clicks.
So how did the pristine world of email advertising get overrun by auto clickers?
Spamming and malicious emails have steadily evolved over time.
The first serious email viruses started in early 1999 with Happy99 and the Melissa Virus. And as people started to transition to banking online, we saw a surge of phishing emails attempting to steal login credentials. But the times of simple phishing are long behind us. Now we deal with highly targeted “spear phishing” emails carefully crafted to worm their way into particular companies and systems.
In 2018, 76% of businesses were the victim of a phishing attack. And IBM pinned the average financial cost of a data breach at $3.86 million.
It’s no surprise that companies are taking this threat incredibly seriously.
Industry leaders work at a ferocious pace to deploy tools and use services that scan inbound emails, any attachments within the email, and every link (even the invisible ones.
So, surprisingly, it’s not scammers or ‘bad people’ that are causing the fake clicks. It’s businesses trying to protect themselves from phishing.
All those firewalls and security tools automatically click links to check the destination as they scan inbound emails. And it generates a huge volume of bot clicks.
During peak times, Paved tracks over 10,000 actions (opens/clicks) performed by a bot each minute.
That’s a number that multiplies fast.
Bot clicks in emails are no longer an edge case. On business-related newsletters, they are quickly becoming the majority of the clicks.
There are dozens of companies providing these security tools. If you’re interested in learning more you can read about Microsoft’s ATP (Advanced Threat Protection) or Barracuda’s Intent Analysis – Inbound Mail
Why click bots are such a tricky problem to solve
First, you need to understand how clicks are tracked within emails.
Before sending an email, your ESP will replace all links within the newsletter to point to their servers. This enables your ESP to route all traffic through their servers before redirecting to the correct URL. This process is fairly simple and works well. However, it has one drawback, as the final URL is not available in the email, email scanning bots have to simulate a real user clicking to see the final URL.
These bots are designed to be indistinguishable from users. It’s what prevents phishing attacks from detecting the bot and serving an alternative page. You can get a glimpse of the sophistication from a Barracuda’s patent detailing the system they developed to check redirect links.
If it was easy to detect bots, hackers could easily conceal their nefarious links and pass the security checks by redirecting the bots to harmless pages.
But the team and I are pretty hardcore about figuring out the complicated stuff.
So we did what we do with everything — we tracked the hell outta it and then figured out how to analyze the information with brain power, and later, AI tech.
Data that proves it’s a bot and not a human
Over the last year, we analyzed bot traffic across hundreds of millions of email clicks. Each tool performs slightly differently — some will scan the email as soon as it is received and some only scan when the recipient checks their mail.
All mainstream bots had one thing in common. They identify themselves as a regular internet user and, if it is an on-prem solution, the IP address is usually the same as the recipient.
One of the most telling attributes that a user is a bot is a lack of engagement on the landing page (the user doesn’t scroll or click anything after clicking on the ad).
These are some of the most telling behaviors of a bot. But on a daily basis we use several dozen attributes to correctly identify and fingerprint a bot.
Our system can tell with a high degree of certainty if the click is from a person or a bot.
How we eliminated wasted ad spend on our newsletters
We verify every click on our Ad Network (our targeted CPC platform). Every single click you get there is a real person.
We also remove automated clicks from the verified performance stats on the profile pages in the Paved Marketplace (where advertisers can book traditional newsletter sponsorships).
While there’s no stopping bots from clicking on a regular newsletter sponsorship, we can make sure that the average CTR stats are pure. On the marketplace, advertisers pay a set cost per impression, so a couple extra bot clicks along the way won’t affect the cost of the sponsorship.
How to stop paying for clicks from bots
If you’re evaluating publishers outside the Paved Marketplace, assume the engagement will be much lower on business-focused newsletters. And adjust your budget accordingly.
If you’re paying per click for email campaigns, ask your publisher or network how they are filtering bots. If you’re seeing low engagement on your landing page, assume their filters aren’t removing automated clicks.
If publishers don’t continually scrub their lists for inactive subscribers, the number bots will start to take over. List quality is as important as ever if you want to get a positive ROI on email marketing.
Paved will release more information and tools around automated clicks and true sponsorship performance soon. Create an advertiser account to get our newsletter or follow us on twitter for updates.