Twitter Queries and the Telephone game

Last week, during the Twitter Chirp conference, Evan Williams apparently (I wasn't there) said that twitter gets about 600 million queries per day. Danny Sullivan claims to have caught Evan after the talk and Evan confirmed that the number was around 19 billion queries per month.

Danny then takes this number and made a blog post about this on Search Engine Land: Twitter Does 19 Billion Searches per month, beating Yahoo and Bing (sort of). In this post, Danny compares the 19 billion number to comscore numbers on google/yahoo/bing worldwide queries. He follows up by pointing out the "(sort of)" part, the caveats as I read them:

• The comscore numbers are third-party, twitter is self-reported. Apples and oranges.
• The number doesn't count partners like Google and Bing.
• The "vast majority" of the searches are API requests, via applications running standing queries.
• Many queries are generated by widgets placed on web pages that automatically return search results to display.

Danny even adds that Evan mentioned that twitter search itself only contributed to the overall queries in "the low double-digits". I'm not sure if that means 10s of millions per day or something else

Danny's article seems completely accurate to me, although certainly suggestive of a comparison that isn't entirely accurate. However, few people seem to read closely to the caveats. People have started reporting Danny's numbers verbatim without mentioning any of the caveats. For example:

Twitter Chirp: Twitter Search is Bigger than Yahoo and Bing Combined

Daily searches on Twitter Should Reach One Billion Queries in May

People have started quoting the articles that are quoting Danny, the notion that twitter is a bigger search engine than bing/yahoo seems to be gaining popularity. I believe that I even heard someone at SMX yesterday claim the 600M/19B numbers were search.twitter.com queries, and that the actual API queries far exceeded this.

My theory though is that the 600M/19B numbers refer to a backend query count, not actual user search queries. The comscore numbers are referring to actual queries done by users in a search box. If a running Tweetdeck instance is monitoring a query every 5 seconds or so, that one instance could rack up around 20,000 queries/day for each query it is monitoring. Similarly, widgets placed on web pages will generate at least one query per page view on those web pages. That's similar to saying google has bajillions of queries because analytics is installed on lots of webpages. Its even possible that me looking at my twitter stream or @gregable stream is a counted query. Furthermore, if I conduct a query on search.twitter.com and leave the page up, it refreshes every once in awhile - presumably each time it polls for new data, that's also considered a query - so 1 user query on search.twitter.com could be hundreds of queries the way Evan is counting.

Note, I don't claim Evan lied. I don't even claim he did anything deceptive in the way he counted queries. Danny put the twitter queries in context with comscore numbers, not Evan. And Danny was clear to point out the caveats, no blame there either.

I would take a wild guess that apples to apples, if you looked at actual twitter search "queries" in the sense that a user considers a query, it's "low double-digit" counts - say 20 million/day, and that those are actually roughly 10x smaller due to repeated polling - say 2 million / day. That puts Danny's comparison more like:

Google: 88 billion per month

Yahoo: 9.4 billion per month

Bing: 4.1 billion per month

Twitter: 0.06 billion per month

Still a huge number for twitter, but a bit more accurate of a comparison. Am I right? Evan? Danny?

Update: 06/08/2010: Techcrunch quoted Twitter's COO Dick Costolo as saying that twitter has 190 million visitors per month. This casts additional cold water on the notion that twitter actually gets 19B user queries every month. It would be strange indeed if the average visitor actually made 100 queries/month on twitter.