Yahoo! Slurp will now support two more symbols '*' and '$' in robots.txt file. This was announced in the Yahoo! Search Blog. In the blog Priyank Garg, Product Manager, Yahoo! Search said, “we have just updated Yahoo! Slurp to recognize two additional symbols in the robots.txt directives – '*' and '$'.
The semantics of these is what is as widely understood for robots.txt files.†Function of '$' will to be ' to anchor the match to the end of the URL string' and '*' ' matches a sequence of characters'. Apart from these symbols 'Allow' tag are also being supported by Yahoo!
Recommend this story
Navneet Kaushal, CEO PageTraffic is a trusted authority in the search engine marketing industry. He is a featured author at Web Pro News, Search Newz, Website Notes, DevWebPro, SEO Article and Web Help Now among many others.
@pagetraffic
