The Yahoo! Blog has announced a weather update for Yahoo! Search. Recently, there were even some assumptions of some possible weather update coming soon. However, this time Yahoo! search engine has supported the X-Robots-Tag directive. This directive basically gives Webmasters the ability to define robots.txt like rules within the http header as opposed to just the META data in the html pages. Google already supports this directive.
The step is taken by Yahoo! To simplify webmaster's control and monthly weather update. To take advantage of this new feature, one needs to simply add the following page level tags to the X-Robots-Tag directive in the HTTP Header. Have a look at some of the examples:
- X-Robots-Tag: NOINDEX — If you don't want to show the URL in the Yahoo! Search results.
- X-Robots-Tag: NOARCHIVE — If you don't want to display cache link in the search results page.
- X-Robots-Tag: NOSNIPPET — If you don't want to display summary in the search results page.
- X-Robots-Tag: NOFOLLOW — If you don't want Yahoo! to crawl links in the page.
Along with this changes, Yahoo! Team is also working on additional changes for crawling, indexing and ranking algorithms.