Alden Bates' Weblog

I noticed some odd things about the hits supposedly coming from Microsoft's Live Search recently. First there's a hit from msnbot for page X. This is followed by another hit on page X, from a Microsoft IP address with the referrer set to a search.live.com results page and with an IE7 user agent, however the second agent only loads CSS and javascript, no images.

Despite the referrer, my web page doesn't appear in the results for the search term given. One hit, for instance, was for TSV 48 for the search term pertwee, yet TSV 48 isn't listed on that search results page (it's on about page 11).

The only clue was a parameter on the referring URL: FORM=LIVSOP. Googling revealed the truth:

msndude says:
The traffic you are seeing is part of a quality check we run on selected pages. While we work on addressing your conerns, we would request that you do not actively block the IP addreses used by this quality check; blocking these IP addresses could prevent your site from being included in the Live Search index.

While there are a lot of complaints in that thread, what Microsoft appear to be doing is testing for spammy pages set up to redirect non-bot visitors to other, less desirable pages. It seems to me that there's a lot of blaming of Microsoft, when people should be blaming the spammers. I fully approve of MS keeping their results free of spam.

Edit: Oh, and Google seem to be doing it too.