On Alexa, Compete.com, Quantcast, et al.

More recently, upstarts that don't seem too dissimilar to Alexa have come along: Compete.com, Quantcast, etc. You can count *dozens* of respected industry folk who have been willing to say stuff like "Compete.com is cooler," "compete.com is better," etc.

So, here's the deal. OK, it's not the deal, it's just my opinion. (Judysbook and Compete.com have "deals," and to be honest I'd rather see them focus on content and metrics, respectively.)

Based on the evidence I've sifted through, there's not a shred to suggest that Compete.com is better at this stage, and some to suggest it's actually worse. That shouldn't be surprising. Going strictly on toolbar installs, the upstart service is bound to have less data, and less representative data. They try to augment that with "ISP relationships," I gather, a practice which is quite old and generally not well explained. (Are Hitwise's ISP relationships better than Compete's? Are Wordtracker's better than KeyWord Discovery's? Does anyone do anything transparently in this industry?)

The CEO of Quantcast, Konrad Feldman, was good enough to talk to me a couple of months ago. I'm still uncertain about where the service is headed, given its newness. Essentially they attempt to combine various data sources to arrive at more accurate rankings, including demographic information for a site. To aid this, sites are encouraged to download some javascript code. Interesting, but no different from the scores of competing analytics services hoping to do the same. Who can get every site or even a lot of sites to install that code?

Evan Williams had a look at Quantcast and found that his site's rank skyrocketed from 15,500 to 2,900 after installing the code. What's wrong with this picture? Doesn't it indicate one or the other number is wildly inaccurate?

A number of analysts, from Greg Sterling to John Battelle, have given Compete.com favorable comments. But the uber-sceptical Matt Cutts just says no. (Check out the epic comments there, with everyone from the Compete.com founder to Lawrence Coburn, to one of the founders of Direct Hit(!) weighing in. The pro-Compete comments sound to me like empty cheerleading, but maybe that's because I'm not.)

And then there's SEOmoz's exhaustive (though restricted unfortunately to the SEM blogger community) attempt to look at the relationship between third-party traffic estimates and the "actual" traffic numbers of the site. This was so thorough that it got into the correlation coefficient between potential indicator of traffic, and actual traffic. Technorati links won as the best predictor. My caveat on this study would be that the "actual traffic" numbers are not reliable enough to give an accurate read - they are not "actual," but rather reported based on whatever log analyzers or other tools the sites use. Most of the site owners are savvy, but that doesn't make them perfect, and their chosen analytics providers are neither perfect nor consistent across the many sites in the survey. Nonetheless, there is something interesting in this study. It would be nice to see it replicated, with slightly more rigor, in different verticals. I found the fact that the third best predictor of actual traffic was the number of inbound links identified by Yahoo Site Explorer to be pretty interesting. Apparently, links matter.

And how! Timely, isn't it, that Google has just weighed in with the availability of a comprehensive linkage report, within Google Webmaster Central (formerly Google Sitemaps). Let's call it the "full linkerooni" - not to be confused with the publicly-available sample available with the "link:" operator. Incredibly extensive account by Danny at SearchEngineLand. Various versions of this might turn out to be topnotch predictors of actual site traffic.

So: some services that aren't really intended to be traffic measurement services predict actual traffic far better than those who make this claim for themselves, it appears. And Compete.com, in SEOmoz's study, comes in below Alexa, which doesn't fare too well in its own right. Shouldn't our cheerleading, then, be directed at Technorati and Yahoo for their fine free tools, and not mysterious startups who have yet to show us anything concrete?

Those in the driver's seat user-tracking-wise are the toolbar (or related user accounts) meisters. No, I don't mean the spyware and low level toolbar hawkers - I mean the respectable guys users hand over their browsing habits to, willingly. The king of toolbars: Google. Others in that vein, like Yahoo. The browser makers, and the like, also. Microsoft, on many levels. Companies that look even a bit like that, have scads more data than any of these "please install our code or download our toolbar so we can get more accurate data" startups.

So: we have panel-based measurement; toolbar-and-browser user tracking; ISP data; javascript code installs; and probably a few other methods I'm forgetting for third-party analysts to get ahold of information about sites they don't personally own, and aggregate rankings and other user behavior data. And as the SEOmoz study shows, if you're not fussy about what a service calls itself - Yahoo Site Explorer is just a private area for webmasters and SEO types to understand site status, for example - there are some great indicators out there that don't even claim as much.

Depending on who owns what info, some self-proclaimed "better traffic measurement" services will continue to provide wildly inaccurate data. Supporters will defend them by saying "sure, but it's free."

Free, as we know, can be costly. Om Malik still threatens to walk out of a meeting for a smoke break if you tout your startup using Alexa stats. Even if he quits smoking, you get the feeling the threat is still good.

Author: Andrew Goodman