Over at the Search Engine Journal, the resident writer Ann Smarty has written an extremely interesting post about how Webmasters can increase the Google Crawl rate for their websites. It is every important to understand one simple concept and it is that, you can never force or cheat Googlebot into crawling your website. But you can always employ certain steps, that would ensure that Googlebot gets attracted to your content and crawls it. Here are a few ways to do just exactly that:
- You should always update your content as soon as possible, because this effort really helps. You should update or add new content to your website at least 3 times a week (minimum) and then should immediately 'ping' (notify) Google.
- Page load time is another critical factor to be taken care of. If the crawler spends too much time crawling your huge images or PDF files, there will be no time left for it to visit your other pages.
- Double check your internal link structure. That means keep an open eye for any duplicate content that might arise via different URLs.
- It is imperative that your server should always return correct header response.
- Another great way to invite Googlebot is to assign unique title and meta-tags for each of your pages.
For full list, visit SEJ
Following these steps, would make your content look attractive to Googlebot and will compel it to crawl your website.
Recommend this story
Navneet Kaushal, CEO PageTraffic is a trusted authority in the search engine marketing industry. He is a featured author at Web Pro News, Search Newz, Website Notes, DevWebPro, SEO Article and Web Help Now among many others.
@pagetraffic
{ 4 comments… read them below or add one }
How do I slow down the bot.
Google is hammering my sites. And my content doesnt change that often.
To slow down the bot, you need to have your site submitted and verified in Google Webmaster tools. There you will get an option to slow the bot.
hey,frns check this trick if you want to increase your google bot crawl rate::
http://latesthub.in/2010/12/05/trick-to-increase-…
hey,frns check this trick if you want to increase your google bot crawl rate::
http://latesthub.in/2010/12/05/trick-to-increase-…