MSNBot Ignores Robots.txt

| 2 Comments
I think I might have accidentally started the MSNBot Boycott when I made some comments on on JWZ's blog. I posted the exclusion rules to put in your robots.txt, to keep Microsoft from crawling your site. I set it up on my site immediately, but I discovered that the MSNBot ignores the robots.txt rules, they scanned my site anyway. MS claims it's a bug. Yeah right. As usual, Microsoft thinks the rules don't apply to them.

2 Comments

Actually, I'm *not* JWZ. He's more famous than I am. :-)

Sorry for the inadvertent misattribution, Jeremy.

Leave a comment

About this Entry

This page contains a single entry by Charles published on July 6, 2003 1:08 AM.

Exercise Your Patritotism was the previous entry in this blog.

Tech Clairvoyant is the next entry in this blog.

Find recent content on the main index or look in the archives to find all content.

Archives

Pages

Powered by Movable Type 5.13-en