Every now and then I jump over to Live Search to see if it’s any better or that it’s better than Google or easier to use or something of the kind. I’m open to lots of possiblities.
Today was one of those days. And I was squarely disappointed.
I won’t go into all the different searches I tried, but let me just showcase one to illustrate a terrible, terrible problem in Live Search: search quality of new information is poorly handled. From my little experiments, if you’re searching for some official Microsoft information, you might find some good links for msdn blogs and the like, but other than that, I’m perplexed as to why the search engine is performing so poorly.
Here goes. This week I’ve been following the Mix09 conference closely from a distance. One of the topics I’ve been keenly interested in is SketchFlow. I’ve been following mention of SketchFlow on Twitter via tools like monitter.com, Twitter itself and FriendFeed. I’ve also been using Google. However, my blog posts about SketchFlow and how Blend works with the Tablet PC have been inching up in Google and since I’m looking for commentary other than mine, I’ve been skipping past my posts.
So today, I thought: Why not look for SketchFlow in Live Search. So I did. The results were not bad, but I did notice right away that my blog post wasn’t shown on the first page, unlike Google’s. So I checked the 2nd page of hits. Nope. Third. Nope. Not that I really wanted to see my blog posts on SketchFlow listed at the top of Live Search, but I did find the missing links curious.
So I decided to do a search for “SketchFlow Tablet PC” together. Surely that would bring up my blog posts since I’ve been the only one I know blogging about the two together. And this really blew my mind. What was the very first hit? My blog post on Sketch flow and how it fits in with the Tablet PC? No. Nope, the first hit was a Channel 9 interview about Tablet PCs from 2004. And the 2nd link was to another Channel9 posting about the Tablet PC Eduction Pack from 2005.
You’ve got to be kidding me!
What was curious was that no direct link to my blog posts on SketchFlow appear at all in Live Search. There are some indirect links and a reprint on WhatIsNew, but that’s it.
Of course, one poor search doesn’t mean that Live Search is bad, but it sure reinforces my skepticism that the group is on the right track.
My engineering sense is suggesting that there’s a fundamental problem going on here. It appears that the rankings are showing Microsoft bias number one and number two do not reflect timely information. That’s not a good thing–at least in terms of how I use search engines.
Now maybe the approach works well with reference information; is that the intent? I wonder if the Live Search team has become so sensitive to search quality–trying to produce the best measurable results over the last what three or so years–that they’ve effectively averaged out new or temporal information that does not fit their “model” if you will. In other words, they’ve tried too hard to get the right answer.
It’s also a possibility that my blog is simply not being included in Live Search any more for any number of reasons. Who knows.
I have no idea how the ranking works, but my intuition is telling me that someone is trying to tweak too much quality out of the ranking. It’s not going to be there. The “right answers” are more temporal for whole classes of searches. I suggest the team spend more time on search.twitter.com or some other real-time search engine and begin to see and relax with the variability of the results.