MSNBot Ignores Robots.txt

| 2 Comments
I think I might have accidentally started the MSNBot Boycott when I made some comments on on JWZ's blog. I posted the exclusion rules to put in your robots.txt, to keep Microsoft from crawling your site. I set it up on my site immediately, but I discovered that the MSNBot ignores the robots.txt rules, they scanned my site anyway. MS claims it's a bug. Yeah right. As usual, Microsoft thinks the rules don't apply to them.

2 Comments

Actually, I'm *not* JWZ. He's more famous than I am. :-)

Sorry for the inadvertent misattribution, Jeremy.

Leave a comment