So who’s doing all this searching anyway?
Written by Jeremy Crane (contact - e-mail) -- November 7th, 2007 | Recommend ThisFor the past few weeks I’ve been peeling back the layers on the search onion to understand a few questions. How effective are each of the engines? How do people use search? Who is using search? I’ve shared a few tidbits of these musings on the blog including a post on “search fulfillment” that raised a few hackles and more recently a post on “navigational search.” In the interest of peeling back another layer in the continuing quest I thought I’d look into exactly who is performing the highest volume of searches on each engine. The beauty of Compete’s people based approach to click stream data is that I can dig into this question without having to worry about things like bots, meta-search, etc. Every search query we see in our data is a query performed by an actual person.
To start I decided to take a look at the overall search query landscape with respect to volumes of queries performed by the US online population. As we are all well aware the online world is littered with examples of the now infamous “long tail.” Search query volumes are no exception.
We ranked individual searchers by the volume of queries performed in a given month and then aggregate their searches into percentiles. What you find is that the top 1% of searchers performs a full 13% of all searches in a given month. If you extend this to the top 20% the number of queries increase to roughly 70%. So in contrast to the standard 80-20 pareto it appears that in web search there is roughly a 70-20 distribution. So what if we break this out by engine?
It appears that high volume Google users are the least concentrated with the broadest distribution as compared to Yahoo! and MSN/Live. 70% of search queries in September were performed by 20% of Google searchers. For Yahoo! the concentration increased to 73% and for MSN/Live searchers 75% of the queries were performed by the top 20%.
Next question … are these high volume searchers the same every month or is there some churn amongst these top searchers?
From this perspective the chart flips, with Google showing the highest concentration of top 20th percentile searchers returning to the top tier in a subsequent month. Yahoo! had roughly 58% of top tier August searchers returning in September. MSN/Live saw only about 52% of the top tier returning. I’m curious about how this looks over the course of a longer period. Do the same 52% to 60% of people continually perform the lion share of searches on the top engines or is there some trend in the data? Does seasonality play a role? So many questions … perhaps for another post.
Did you like that post? You'll love these.
Done reading? subscribe: To get an automatic feed of all future posts subscribe here, or to receive them via email enter your email address in the box in the right column.
Link to This Post:
Posted under: Online Media & Search |
Trackback Link | Permalink | 
Get the 
November 7th, 2007 at 3:03 pm
Actually, in the Long Tail concept depicted by Chris Anderson, “products that are in low demand or have low sales volume can collectively make up a market share that rivals or exceeds the relatively few current bestsellers and blockbusters, if the store or distribution channel is large enough”.
Consequently, the Long Tail effect does not apply to search queries : top searchers account for a too important part of the queries.
Do you agree?
November 8th, 2007 at 2:38 am
What is the effect of http://club.live.com/ to Live Search results in this data? Live Search experienced a noticable bump when it introduced the Live Search Club last summer, and it would seem to fit in the mix: a large number of searches done by a small percentage of users, and a noticable dropoff (after prizes were achieved?).
November 8th, 2007 at 3:48 am
In response to the article I’d have to go with Google as the most proficient search engine. I myself do A LOT of SEO and web page diagnosis. As far as algorithms are concerned Google is a quad core while Yahoo and MSN are Pentium 4s… In the grand scheme they get the job done. but if any “best sellers come out” you might have to wait a week before msn or yahoo indexes them. If you’re looking for a good price you’ll have to wait a month before anything appears…
In response to the comments I’m not sure if I agree… I think the statistics are definitely something to look at and consider when search engine optimizing; but to say that a “best seller” can out perform something in “less demand” would vary depending on the product and the search engine.
I help with marketing for a national dining guide dinersnation.com ( http://www.dinersnation.com ) and I’m constantly adjusting SEO; in retrospect if a site that marketed things that were not in high demand they could still out perform guys who readily update and sell high demand items.
I’ll use my particular site (a national dining guide), http://www.dinersnation.com as an example. I have 3000 some odd restaurants in my database, in a profile system similar to myspace; granted many of these restaurants have been in existence for many years (not high demand), and many of them have their own website (this makes my job even harder).
When an experienced web designer or marketing director designers advertising and/or a web page, it’s important to look at both the long tail and short tail hit from the consumer.
In my case, a potential viewer looks for 1 of 2 things:
1. (short tail) - The EXACT name of a restaurant (I’ll say this is the “hot new item” which is one of the hardest to compete with)
or
2. (long tail) - Something in the category of the restaurant. (think of this like restaurants in a particular area or that fit this criteria, only for our example it would be a product that relates to, or meets a particular spec or description)
in this case the short tail or “hot item” would be easily found in the top 10 pages of any search engine and theoretically outperform the out-of-demand item.
but here is the science behind all this madness.
The short tail hit typically has 1 to 2 possible search terms to find a particular site. in my case the exact name of a restaurant (which will usually lead them to the restaurant’s homepage)
BUT
for a long tail hit result you have a multitude of possible search terms. In my case, 50. There are 50 general search variations in order to find just one profile (product) on my site. Not including all the random and weird variations of words or phrases that could accidentally send someone stumbling onto a page.
November 8th, 2007 at 9:38 pm
It would be interesting to see what percentage of those TOP searchers click on the sponsor links
and what percentage click on the organics first.
Furthermore, it would be interesting to see what was the average final position they clicked on until they are satisfied with the answers they had attained
November 20th, 2007 at 6:20 pm
Great post! Looking forward to the follow-up post because, as you suggested, there are lots of fascinating questions raised. Like most great analysis, new WHATs lead to new WHYs.
Regarding whether the trends hold true over time, perhaps you can look at “decile mobility,” that is, the percentage of today’s top 10% of searchers show up in a different decile in the next time period.
Thanks again…
January 2nd, 2008 at 12:28 pm
I’m curious about the repetitive nature of search - I find myself lazily redoing a search sometimes several times on Google - often because I’m multi-tasking or interrupted. It would be interesting to understand this across the board, and comparatively on the major engines.
Any insight?