Why your web search results and mine are not the same

I recently asked, on the MHSLA listserv, for some help with finding a web-based resource about which I had very little information. I turned up a few possibilities; my colleagues, also running web searches, directed me to other sites that I had not found. While it’s likely that some of the difference in what we found lies in the varying approaches each searcher may take to answering a question, Marydee Ojala’s presentation “Searching, Finding and the Information Professional” (Computers in Libraries 2007 Powerpoint presentation) suggests five reasons that, even when we do key in on the same search terms, we may see very different results.

  1. Different search engines work from different databases. While 64% of all web searches are done in Google, searchers who use Ask (4%) or MSN (8%) or Yahoo (22%) will see results that don’t show up at the top of the Google results list, if they show up at all. (Statistics from Hitwise. I found them in Google.)
  2. Web search engines track us, and learn about us as we use them. They know where we are, and try to find materials that match our search history. Consider this: when I go to Amazon.com at home, the first screen recommends anime and science fiction titles. At work, it recommends health and hospital titles. Even when I’m not logged in! Web search engines also skew the results based on past use at a particular location. Imagine what this does to the searches at the library’s public-access workstations.
  3. “Squishy Boolean.” This concept, described by Mary Ellen Bates in 2005, includes the use of “preferred but not required” terms in addition to the standard AND/OR/NOT logic. Bates also describes some search engines that allow weighting of terms to indicate how important a particular term might be. I don’t see an obvious way to use “squishy Boolean” in any of the “big 4” search engines (Google, Yahoo, MSN, Ask) right now, but Bates wrote that she’d seen it in beta. (Bates ME. Squishy Boolean. Online 2005 Mar-Apr; 29(2): 64. Michigan readers, use this link to find the article in Gale.)
  4. Algorithms taking over. Sometimes, the search engines just ignore our carefully structured advanced-technique Boolean, and run the search against their default algorithms instead. I have seen this myself on a recent Ask search; a term I had specified should not be present turned up in the first five hits.
  5. Search Engine Optimization (SEO). Content providers use a number of techniques to increase the likelihood that their sites will show up when someone searches a related topic. Some web information services even “game the system” to have their results show up first. Ojala suggests a way to minimize the effect of SEO on how we see our results: use the “preferences” settings in the search engines we use to set the “number of results per page” as high as possible; it’s faster to scan 100 results on one page than to keep turning the page for the next ten … next ten … next ten …