Bing now has a new crawler. The new Bing crawler user agent string listed as:
Note that the bot’s name for Bing didn’t change. You’ll also see the previous version of the bot as well:
— which is still in use, as of today, as Bing’s primary user agent.
Here are some Question and Answers that probably can help you!
Q: Bing is crawling my site, so why is it not indexing my pages?
A: Make sure your content is viewable by MSNbot.
- Ensure there are text navigation links if main navigation is pictorial-based
- Ensure there is plenty of indexable content outside of <script> </script>tags
- Make sure each page has unique <title> and <meta> description tags
- If possible, avoid frames and meta tag refresh
- If your site uses session IDs or cookies, make sure they are not needed for crawling
- Use friendly URLs whenever possible
- Review our Webmaster Center blog post, I’m not ranking in Bing, what can I do?
Q: My site is moving. How do I set up a redirect?
A: In most instances, you will want to use a 301 redirect . There are many different ways to institute these redirects, based on your preference and the technologies your site is using.
Q: Why isn’t Bing crawling or is only partially crawling my site?
A: Depth of crawl is based on a lot of factors, including how your site ranks and the architecture of your site. However, here are some steps you can take to help us crawl your site more effectively:
- Verify that you are not blocked and there are no site crawling errors in Webmaster Center tools
- Make sure you are not blocking access to the Bing web crawler, MSNbot, in your Robots.txt file
- Make sure you are not blocking MSNbot’s IP range
- Ensure you have no site navigation issues that might prevent MSNbot from crawling your site:
- Invalid page coding (HTML, XHTML, etc.)
Note: You can validate your page source code at www.w3c.org.
- Password protected pages
- Submit a sitemap in the Sitemaps tool of Webmaster Center tools and provide the path to the sitemap.xml file(s) in a Robots.txt file