search engine query

Do any of you use MSN as your primary search engine? The MSNbot that crawls my site is pretty much just stealing bandwidth, so I’m about to ban that bot from crawling here. For comparison purposes, allow me to present you with a few statistics from the month of June 2005.

    Exhibit A – MSNbot

  • Crawl hits = 9561
  • Bandwidth used = 124.43 MB
  • Visits to as a results from searches @ Google = 683*
    Exhibit B – Googlebot

  • Crawl hits = 3415
  • Bandwidth used = 51.74 MB
  • Visits to as a results from searches @ MSN = 41

The cost/benefit just isn’t enough for me. Not that I’m starving for bandwidth, and not because it’s causing performance issues (a href=””>Go Mindstorm Hosting!), but it just annoys me. And since it’s my site, I can do what I want right?

So, effective immediately:

User-agent: MSNBot
Disallow: /

If you notice any errors anywhere or your access gets blocked for some reason (I’m blocking some other bots and things as well), please email me and let me know.

* Includes both standard web searches as well as Google Images searches.