« Exercise Your Patritotism | Main | Tech Clairvoyant »

MSNBot Ignores Robots.txt

I think I might have accidentally started the MSNBot Boycott when I made some comments on on JWZ's blog. I posted the exclusion rules to put in your robots.txt, to keep Microsoft from crawling your site. I set it up on my site immediately, but I discovered that the MSNBot ignores the robots.txt rules, they scanned my site anyway. MS claims it's a bug. Yeah right. As usual, Microsoft thinks the rules don't apply to them.

Comments

Actually, I'm *not* JWZ. He's more famous than I am. :-)

Sorry for the inadvertent misattribution, Jeremy.

Post a comment

(If you haven't left a comment here before, you may need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Thanks for waiting.)