Yahoo! Adopts Robots.txt Tag To Block Specific Content

May 28, 2007 | 2,194 views | by Navneet Kaushal
VN:F [1.9.20_1166]
Rating: 0.0/5 (0 votes cast)

Over at Search Engine Land a post says Yahoo! has adopted the Robots nocontent tag whose function is to block a certain part of your Website which you don't want to be included in an index.

"The new robots-nocontent tag now allows you to tell Yahoo to ignore the clutter. Simply use the tag (technically, it's an attribute) to surround text you do NOT want included in searchable content within Yahoo."

Priyank Garg at Yahoo's official blog writes, "This tag is really about our crawler focusing on the main content of your page and targeting the right pages on your site for specific search queries. Since a particular source is limited to the number of times it appears in the top ten, it's important that the proper matching and targeting occur in order to increase both the traffic as well as the conversion on your site. It also improves the abstracts for your pages in results by omitting unrelated text from search result summaries." The reason for the latest move being “we saw that it was compatible with existing standards and microformats and that makes it easier to gather broader support, including from the other search engines.”

Read "Search Illustrated: Blocking Search Engines With Robots.txt" for a comprehensive understanding of functions and usage of robots tag.

4.thumbnail Yahoo! Adopts Robots.txt Tag To Block Specific Content

Navneet Kaushal

Navneet Kaushal is the founder and CEO of PageTraffic, an SEO Agency in India with offices in Chicago, Mumbai and London. A leading search strategist, Navneet helps clients maintain an edge in search engines and the online media. Navneet's expertise has established PageTraffic as one of the most awarded and successful search marketing agencies.
4.thumbnail Yahoo! Adopts Robots.txt Tag To Block Specific Content
4.thumbnail Yahoo! Adopts Robots.txt Tag To Block Specific Content