Today, Ask.com, Google, Microsoft Live Search and Yahoo! together announced support of “auto discovery” of Sitemaps. The new open-format autodiscovery permits website owners to specify the location of their Sitemaps within their robots.txt file. This altogether eliminates the need of submitting sitemaps to each search engine separately.
Just add a line like this one in your robots.txt file:
Yahoo! Official blog says:
All search crawlers recognize robots.txt, so it seemed like a good idea to use that mechanism to allow webmasters to share their Sitemaps. You agreed and encouraged us to allow robots.txt discovery of Sitemaps on our suggestion board. We took the idea to Google and Microsoft and are happy to announce today that you can now find your sitemaps in a uniform way across all participating engines.
Comprehensiveness and freshness are key initiatives for every search engine, and with autodiscovery of sitemaps, everyone wins:
- Webmasters save time with the ability to universally submit their content to the search engines and benefit from reduced unnecessary traffic by the crawlers.
- The search engines get information with regards to pages to index as well as metadata with clues about which pages are newly updated and which pages are identified as the most important.
- Searchers benefit from improved search experience with better comprehensiveness and freshness
In addition, Ask.com is now supporting submission of Sitemaps via http://submissions.ask.com/ping?sitemap=SitemapUrl.
Official Google blog says:
We've been busy trying to get Sitemaps adopted by everyone and to make the submission process as easy and automated as possible.
First, we're making the sitemaps.org site available in 18 languages! We know that our users are located all around the world and we want to make it easy for you to learn about Sitemaps, no matter what language you speak. Here is a link to the Sitemap protocol in Japanese and the FAQ in German.
Live search says:
Today, I'm happy to announce the latest addition to the protocol: Sitemap Autodiscovery. Autodiscovery gives siteowners the ability to easily share their sitemaps with all search engines at once, without the overhead of manually submitting them to each one. In order to share your sitemap with the crawlers at large, you simply provide the following line in your robots.txt file: