How to Control the Crawl Rate of Googlebot? – The Latest Video from Google India Webmaster Community!

Nov 15, 2014 | 1,737 views | by Navneet Kaushal
VN:F [1.9.20_1166]
Rating: 5.0/5 (1 vote cast)

A new video in response to #AskSearchQuality campaign has been recently released by Google India Webmaster Community. The campaign is exclusively for Indian webmasters.

The video answers the questions related how one can control the crawl rate of Googlebot.

The Google representative started with explaining what crawl rate exactly is. He said that most of the webmasters often assume crawl rate to be the frequency at which Google crawls the content on website. But, it is actually the speed with which Googlebot interacts with the server to get the content from the website. If you have new content or are updating content that you want to be indexed immediately, you should probably use the "Fetch as Google" because there you have the option to submit to index.

But, if crawl rate is bothering you, there's an option of decreasing the crawl rate but not really increasing it. The only situation where it makes sense to decrease the crawl rate is when you see the Googlebot pinning the server is affecting the website and slowing down the things. Unless you see a substantial change, Google doesn't recommend you to make tweaks. The rate can however be decreased from Webmaster tools. The video also explained the steps on how to alter the crawl rate in the webmaster tools. However, this setting lasts for only a period of 90 days.

Another question addressed at the end was how to allow Googlebot to crawl .css & .js files. To which, the representative said that unless you have disallowed the bot from crawling your .css & .js files from crawling your robots.txt , Google will crawl.

How to Control the Crawl Rate of Googlebot? - The Latest Video from Google India Webmaster Community!, 5.0 out of 5 based on 1 rating
1.thumbnail How to Control the Crawl Rate of Googlebot?   The Latest Video from Google India Webmaster Community!
Navneet Kaushal is the founder and CEO of PageTraffic, an SEO Agency in India with offices in Chicago, Mumbai and London. A leading search strategist, Navneet helps clients maintain an edge in search engines and the online media. Navneet's expertise has established PageTraffic as one of the most awarded and successful search marketing agencies.
1.thumbnail How to Control the Crawl Rate of Googlebot?   The Latest Video from Google India Webmaster Community!
1.thumbnail How to Control the Crawl Rate of Googlebot?   The Latest Video from Google India Webmaster Community!