Research

When Email Open Rates Are Too Good to Be True

How our metrics were almost overstated

Dark Choice OMG card headlined When Email Open Rates Are Too Good to Be True, subtitled How our metrics were almost overstated. Beside it, reported rates against rates without security scanners: opened 19.11 percent and 14.69 percent, clicked 5.21 percent and 0.64 percent.
Share of 169,502 tracked sends with an open or a click, before and after removing security scanner ranges.
On this page

Security scanners inflated our email rates: 14.69 percent of sends opened and 0.64 percent clicked without them, and Apple Mail Privacy Protection is still inside the open figure. 12,866 of our 14,274 clicks came from scanners, a share that held at 88.4 percent after we froze our scanner list. Those clicks had already written to other systems: 2,872 CRM contacts rested on scanner clicks alone, and 83 addresses were unsubscribed without anyone asking. Yours depends on your list, and you can check it without an IP address.

Our email reports showed 19.11 percent of sends opened and 5.21 percent clicked. Without security scanners, those rates fall to 14.69 percent and 0.64 percent.

The reports were the easy part to fix. The same clicks had been feeding our CRM for months: every click created an engaged lead, and when we checked in August, every click behind 2,872 of those contacts had come from a scanner. Only 743 contacts had a click from a person.

The scanners were doing their job. They follow the links in the mail they inspect (Microsoft's Safe Links scans URLs before delivery). Our mistake was treating a click as a person, because anything wired to that event then fired for the machine too. A "clicked before" segment fills up with companies whose mail runs through a scanner, and each send proves the segment right when the same scanner clicks again. We had sorted a list of Canadian businesses by their mail security and called it engagement.

The full count

The CRM was one symptom. Across every tracked send from February to August 2026, the same ranges account for 12,866 of 14,274 click events, or 90.1 percent.

Sends are a fairer count than clicks, because a scanner and a person can both click the same email. A send counts below if any click on it qualifies.

What counts as a click (169,502 sends) Sends Click rate
Anything the platform recorded 8,832 5.21%
From outside the scanner ranges 1,088 0.64%
Also not an unsubscribe link 646 0.38%
Also after 90 seconds, from an IP that touched no other send 361 0.21%

Interest sits somewhere between the last two rows. The bottom row is deliberately harsh: it drops a quick reader, and it drops two colleagues on one office connection. Opens held up better. 8,681 of 40,919 open events came from the same ranges (21.2 percent), which takes the open rate from 19.11 percent of sends to 14.69 percent. That lower figure is still a ceiling, because Apple Mail Privacy Protection does not show up in IP ranges.

Scanners click everything. 2,007 of the 2,040 clicks on phone number links came from those ranges, and nobody places a call through a security appliance.

What had to be cleaned

A wrong number in a report would have been cheap. We had wired this one into other systems, and each needed its own cleanup.

Where What the scanners left behind What we did
CRM 2,872 contacts and 2,810 companies with scanner clicks only Relabelled 2,381 and 2,148, deleted 206 and 183, kept 285 and 479
Accounting 2,425 of those contacts synced as customers Cleaned out
Audience segments 1,507 addresses moved up to "opened before" on scanner opens alone Rebuilt on opens from outside the ranges
Unsubscribe list 83 opt-outs nobody asked for Restored 72, left 11 suppressed
Web analytics 4,471 of 4,882 page views on one heavy send day Flagged at our tagging server

The CRM took the most care, because bulk deleting would have been the wrong fix. 285 of those contacts and 479 of the companies carried a human signal such as a deal or an email thread, so real relationships were sitting in the scanner pile. 2,425 of the contacts had also synced to our accounting system as customers, which is why most were relabelled instead of deleted.

The unsubscribes are the part we like least. Any click on an unsubscribe link, scanner or person, made our sales alert post that URL into team chat. The chat app fetched the URL to build a preview, and the link treated the fetch as a request to leave. 221 of our 743 unsubscribes carried the chat app's user agent. In 138 of them a real person was opting out and the preview got there first. In the other 83, only scanners had ever touched the link, so nobody had asked to go. We restored 72; the remaining 11 had bounce or deliverability problems and stayed suppressed.

The fix has been public since January 2017. RFC 8058 describes this failure in anti-spam software and answers it with a POST. Our link now shows machines a confirmation form instead of acting.

The website counted scanners too, since a scanner loads the page and runs the tag. We pasted the ranges into GA4's internal traffic rules, and they never matched. Behind a server-side tagging server, Google tests those rules against the server's address, so the filter has to live on that server, where the visitor's IP is known.

Testing a list against the data it came from

Every figure above rests on a list we built from the same logs, which is a good reason to distrust it. We added 37 of our 43 ranges on August 21 using four signals: a click under 90 seconds after sending, one IP clicking five or more sends, every link in a send fetched within about ten seconds, and fake or missing user agents. A list built that way will flatter itself. So we ran checks that do not depend on how it was built.

The list stopped changing. It has not been edited since August 21, and 1,898 clicks arrived after that commit. Those ranges produced 88.4 percent of them.

The recipients split cleanly. People at company domains and people on Gmail click links in much the same way. Their mail does not travel the same route.

Recipient Clicks From scanner ranges
Company domains 13,768 92.2%
Consumer Microsoft addresses 230 74.8%
gmail.com 196 1.0%
Other consumer and ISP addresses 80 1.3%

This table does not show Gmail is free of bots. Our ranges are mostly Microsoft (32 of 43) and AWS (9), with none from Google, and Omnivery reports Gmail bot clicks near 22 percent from late February. It shows our list is not simply catching people in offices, because people in offices also use Gmail, and the list barely touches them there.

Late clicks do not rescue the headline. If every click from these ranges after the first hour was a person, the share falls only to 88.0 percent (12,555 of 14,274). Speed points the same way, with a median of 0.5 minutes inside the ranges and 16.2 minutes outside. Speed was also one of the mining signals, so it supports the case without proving it.

The error runs in both directions. 367 of the 1,408 clicks outside the ranges arrived in under 90 seconds, so some scanners are still in the "human" group, and we have no list of confirmed people to test against. We would defend "about nine in ten" and nothing more precise.

The figure also stops at our list. 13,768 of our 14,274 clicks went to company domains. Omnivery puts corporate domains at 80 to 95 percent and consumer email at about 16 percent, and a 2020 M3AAWG paper reported 20 to 80 percent for business senders and under 10 percent for consumer ones. A retailer mailing Gmail addresses should not borrow our headline.

How to check your own list

Start with the test that needs no IP data. Export sends and clicks, split recipients into Gmail and everything else, and compare click rates. Ours read 0.27 percent for Gmail against 7.74 percent for everyone else, and removing the scanner ranges took the second number to 0.83 percent. A gap that wide is your first estimate of the problem.

Then read what your platform already offers, because the defaults differ.

  • Mailchimp turns bot filtering on by default and keeps the underlying data.
  • Klaviyo marks each click with a Bot Click property and lets you exclude them from reporting and attribution.
  • ActiveCampaign leaves BotSense off by default, and it works going forward only.
  • Kit has separate toggles for clicks and opens, and filtered clicks cannot be recovered.

If you hold raw events, this is the bottom row of our table in SQL (Postgres). scanner_ranges is your own CIDR list.

WITH click AS (
  SELECT e.send_id, e.ip, e.url, e.created_at - s.sent_at AS delay
  FROM email_events e
  JOIN sends s ON s.id = e.send_id
  WHERE e.type = 'click'
    AND NOT EXISTS (SELECT 1 FROM scanner_ranges r WHERE e.ip <<= r.cidr)
), reach AS (
  SELECT ip, count(DISTINCT send_id) AS sends FROM click GROUP BY ip
)
SELECT DISTINCT send_id
FROM click JOIN reach USING (ip)
WHERE url NOT ILIKE '%unsub%'
  AND delay >= interval '90 seconds'
  AND sends = 1;

Build the range list from the four signals above, confirm each owner with RDAP, and leave out Apple and residential ISPs. We started with six ranges, which alone caught 70.3 percent of clicks.

Four rules apply whatever tool you use. Nothing automatic should fire on a raw click: no CRM record, no sales alert, no segment. An unsubscribe link should act only after a person confirms it or a mail client sends the POST. A URL that changes something on fetch should never be posted into a chat app that builds previews. And before you delete what a scanner created, check it for deals and replies, because ours hid real relationships.

Then we turned tracking off

September added a cost we had not counted. Amazon SES records opens and clicks by rewriting every link and adding a pixel, and Gmail answered with its suspicious-content banner. We sent four otherwise identical messages on September 9: both tracked copies drew the warning, and neither untracked copy did.

Four messages is a thin test. It was enough, because the trade was a warning on our email against a click column we now knew was mostly noise.

Our next send went out with tracking off. We can no longer segment that audience by who clicked, and the column we gave up had been sorting our list by mail provider.

Method

Sends are our own marketing email to Canadian businesses, through Amazon SES, with click events carrying source IPs. Click events run from February 13 to September 15, 2026, and tracked sends end on August 21. Rates use the 169,502 sends in campaigns that recorded clicks. Recipient groups come from the email domain, so a company on Google Workspace counts as a company. Where a mail provider was on file, Microsoft-hosted addresses showed 95.7 percent from scanner ranges across 2,617 clicks and Google-hosted addresses 20.0 percent across 165. Click and open figures were pulled on November 1, 2026. CRM, segment and unsubscribe figures come from our August 21 cleanup records.

Need Help With Your Digital Marketing?

Request a free discovery call with our team.

Get in Touch