Yahoo! Slurp Supports Two More Symbols In Robots.txt

Nov 3, 2006 | 1,453 views | by Navneet Kaushal
VN:F [1.9.20_1166]
Rating: 0.0/5 (0 votes cast)

Yahoo! Slurp will now support two more symbols '*' and '$' in robots.txt file. This was announced in the Yahoo! Search Blog. In the blog Priyank Garg, Product Manager, Yahoo! Search said, “we have just updated Yahoo! Slurp to recognize two additional symbols in the robots.txt directives – '*' and '$'.

The semantics of these is what is as widely understood for robots.txt files.” Function of '$' will to be ' to anchor the match to the end of the URL string' and '*' ' matches a sequence of characters'. Apart from these symbols 'Allow' tag are also being supported by Yahoo!

1.thumbnail Yahoo! Slurp Supports Two More Symbols In Robots.txt
Navneet Kaushal is the founder and CEO of PageTraffic, an SEO Agency in India with offices in Chicago, Mumbai and London. A leading search strategist, Navneet helps clients maintain an edge in search engines and the online media. Navneet's expertise has established PageTraffic as one of the most awarded and successful search marketing agencies.
1.thumbnail Yahoo! Slurp Supports Two More Symbols In Robots.txt
1.thumbnail Yahoo! Slurp Supports Two More Symbols In Robots.txt