New bot for Bing Search Crawler

bing-search-bot-msn-live

Bing now has a new crawler. The new Bing crawler user agent string listed as:

msnbot/2.0b (+http://search.msn.com/msnbot.htm).

Note that the bot’s name for Bing didn’t change. You’ll also see the previous version of the bot as well:

msnbot/1.1 (+http://search.msn.com/msnbot.htm)

— which is still in use, as of today, as Bing’s primary user agent.

Here are some Question and Answers that probably can help you!

Q: Bing is crawling my site, so why is it not indexing my pages?

A: Make sure your content is viewable by MSNbot.

  • Ensure there are text navigation links if main navigation is pictorial-based
  • Ensure there is plenty of indexable content outside of <script> </script>tags
  • Make sure each page has unique <title> and <meta> description tags
  • If possible, avoid frames and meta tag refresh
  • If your site uses session IDs or cookies, make sure they are not needed for crawling
  • Use friendly URLs whenever possible
  • Review our Webmaster Center blog post, I’m not ranking in Bing, what can I do?

Q: My site is moving. How do I set up a redirect?

A: In most instances, you will want to use a 301 redirect . There are many different ways to institute these redirects, based on your preference and the technologies your site is using.

 

Q: Why isn’t Bing crawling or is only partially crawling my site?

A: Depth of crawl is based on a lot of factors, including how your site ranks and the architecture of your site.  However, here are some steps you can take to help us crawl your site more effectively:

  • Verify that you are not blocked and there are no site crawling errors in Webmaster Center tools
  • Make sure you are not blocking access to the Bing web crawler, MSNbot, in your Robots.txt file
  • Make sure you are not blocking MSNbot’s IP range
  • Ensure you have no site navigation issues that might prevent MSNbot from crawling your site:
    • Invalid page coding (HTML, XHTML, etc.)

Note: You can validate your page source code at www.w3c.org.

    • Site navigation links that are only accessible by JavaScript
    • Password protected pages
  • Submit a sitemap in the Sitemaps tool of Webmaster Center tools and provide the path to the sitemap.xml file(s) in a Robots.txt file
Share your love

One comment

  1. With this new progress, Bing and Yahoo could surely get even with Google. It is really great to have our websites ranked well in this three search engines

Leave a Reply

Your email address will not be published. Required fields are marked *