Crawl Improvements By Yahoo!

Aug 23, 2007 | 1,937 views | by Navneet Kaushal
VN:F [1.9.20_1166]
Rating: 0.0/5 (0 votes cast)

Yahoo! is working hard to impart fresh updates to its “crawl, index and ranking systems.” Recently, a significant reduction on the crawling intensity was noticed as it has now become more efficient and less aggressive. The fresh updates will be relatively important for webmasters.

According to Yahoo! Search Blog:

“In response to feedback from the community, we’re reducing our crawling machines by a significant amount with this update, so you might have observed a significant reduction in crawl load from us along with fewer machine IP addresses hitting you over the last few days. The new crawler continues to improve comprehensiveness and freshness of coverage while also enhancing crawl efficiency, which reduces spurious load on websites.”

Yahoo! experienced an increase in their crawl load while testing and rolling out updates, in support for 'Dynamic URL Rewriting' in Site Explorer. It is working on different internal policies to ensure this doesn't happen, even temporarily, in the future.

Read the whole blog here.

4.thumbnail Crawl Improvements By Yahoo!

Navneet Kaushal

Navneet Kaushal is the founder and CEO of PageTraffic, an SEO Agency in India with offices in Chicago, Mumbai and London. A leading search strategist, Navneet helps clients maintain an edge in search engines and the online media. Navneet's expertise has established PageTraffic as one of the most awarded and successful search marketing agencies.
4.thumbnail Crawl Improvements By Yahoo!
4.thumbnail Crawl Improvements By Yahoo!