Jan 29, 2015 114 reads by Ritu Sharma

Google said to treat Google crawlers, Googlebot, as you would any searcher or user accessing your site from the US. Now Google has initiated a change because it may crawl your site from IP addresses outside of the US and also may crawl your site with language settings outside of English-US.

Google announced that they will now support Locale-aware crawling by Googlebot– which is a huge change for internationalized sites.

Some sites that offer internationalized content have been offering so without sending the user to a special URL. Google has always preferred its users to set up specific URLs or ccTLDs for content tailored to different countries or languages, but most of the sites just serve their content on their .com based on the IP address origin or their browser language configurations.

Google has said that it is now going to support the sites that serve International content based on IP or language. They will do this based on these two methods:-

  1. Geo-Distributed Crawling– Googlebot would start to use IP addresses that appear to be coming from outside USA, in addition to the current IP addresses that appear to be from the USA that Googlebot currently uses.
  2. Language-Dependent Crawling– Here Googlebot will start to crawl with an Accept-Language HTTP header in the request.

Pierre Far, one of the Googlers assigned to this project explained on Google+ a dew points:-

We still (very strongly) recommend having separate URLs for different locales and using rel-alternate-hreflang annotation for them. Separate URLs are better for users, and that's what really counts. Locale-aware crawling is for the few edge cases where it's not possible for you to have separate URLs.

  • It's early days and the countries that Googlebot will appear to come from and the Accept-Language headers it may try do not cover all combinations of countries and languages around the world. Also, we will continue to tweak things as we build out this feature. This is another reason to have separate URLs.
  • Locale-aware crawling gets enabled algorithmically if we detect your site may benefit from it. You don't need to do anything.

There is more information related to this, however, expect things to change slightly as webmasters discover how Google handles this and they report issues, confusion and problems in the forums.

Google Now Supports Locale Aware Content Based On IP & Languages!, 5.0 out of 5 based on 1 rating
Ritu Sharma
Ritu from PageTraffic is a qualified Google Adwords Professional and Content Head at PageTraffic. She has been the spear head for many successful Search Marketing Campaigns and currently oversees Content Marketing operations of PageTraffic in India.
Most popular Posts
Upcoming Events
Events are coming soon, stay tuned!More