We took our headline number down. Here is why.

For nineteen days, from September 3 to September 22, our free inspection led with one number: how many of every hundred visitors had a reason to leave your site. On September 22 we took it down. This post is the record of what it was, what we found when we checked it, and what the report says instead.

What the number was

The inspection engine runs a few dozen checks on a page. Four of them were tied to research about people leaving websites: a slow page, a phone number that cannot be tapped, no way to get in touch, and English only in Southwest Florida.

Each failed check was paired with a published rate, and the rates were combined with a formula that counts each visitor as leaving at most once.

The arithmetic was sound. We wrote a whole page defending it, and that part still holds. A number is only as good as what goes into it, and that is where it failed.

What we found when we read the four studies

Until September 22 the engine cited each rate by name and year. That morning we read each study at its source and printed, beside every rate, who was asked or what was measured. Here is what that reading found.

None of the four was a fair rate to apply to the visitors of a business we had never studied. One was disowned by its own side. One measured a different thing from the thing we checked. One was what people said, not what they did. One was a map of who lives nearby.

What our own data said

A number that can only take a handful of values, and that gives more than half of all businesses the same one, is not much of a measurement. It looked like a measurement. It behaved like a label.

What the report says instead

The four checks stay. They are measurements: the engine timed your page, it looked for a tappable number, it looked for a way to reach you, and in Florida it looked for a Spanish version. What changed is the sentence under each one. It now says what was measured on your page today, and it attaches no borrowed number to it.

The Spanish check now says only what the Census can say. In Lee and Collier counties, about one resident in nine speaks Spanish at home and English less than very well. That is a market a page does not reach, not a count of anyone who left.

The report quotes that figure to every Florida page with no Spanish version. It is for those two counties only, and other Florida counties differ.

The report leads with the grade and the findings. We have run it on 78,807 real business websites (how we count them, and the public file).

How to weigh a number, ours or anyone's

• Ask who was asked. A rate from 262 business buyers is a fact about 262 business buyers. It says nothing about a taqueria's customers until somebody shows that it does.
• Ask whether the condition matches. A study about a phone number missing from a search result is not a study about a phone number on a page that cannot be tapped.
• Ask whether the source still stands. A figure its own author has withdrawn is not a figure you can look up.
• Ask what was watched. What people say they would do and what a count of visits shows are different kinds of evidence. A report should say which one it is holding.

We wrote those four questions after failing our own test on all four. They are printed here so you can hold us to them.

Your own report takes a few seconds. Run the free inspection: it leads with the grade, and every finding says what it looked at, so you can check it yourself.