Blocking your Redirect 301 with robots.txt file? Bad Move!

May 30, 2008 | 4,440 views | by Navneet Kaushal
VN:F [1.9.20_1166]
Rating: 0.0/5 (0 votes cast)

If you are thinking of blocking your redirect 301 with a robots.txt file, then you would be making a big mistake. According to Search Engine Roundtable, there is a thread at Google Groups that is concentrated on this very concern.

One of the users who probably started the thread took up this topic in a very long post and then finally got a response from Googler JohnMu. According to the user, he moved from to, on a domain name sale, but wanted to retain his links, so set up a 301 redirect from .com to .info for a certain period of time. Here is the user's post:

“I recently moved domains from (buyer has since turned
it into an ad portal) to

All the content has been moved to the new domain. I submitted a request to remove all pages several weeks ago, and you can see this is true:

Google has indexed a significant part of my site

including the first set of key pages that you see for the above.

However, when you do a

many of the links show up as links to not to!!! The fifthe entry, "berhomepage" .. go there, and
look at the two insulator links at the way bottom… The go to !!!

I think Google is confused. I suspect my queries are suffering, or in
any case, the link: is misleading at least me!

How do I get link: to report the proper links to my new site so I can
monitor my efforts to get other sites links changed over to my new

If there is any questions, returns HTTP/1.1 301 Moved
Permanently to, and I have set the Google
preference to

Sorry for the extra explainations… I'm afraid I'm getting penalized
in a way I don't know, because of the (old) presence of

I used to be the top ranked for a number of searches
insulators, glass insulators, telephone insulators, etc.

Now, I don't even see my site in the rankings… well, top 50 like for
glass insulators.”

And now Googler Mu's response:

“Hi Mr. Hemingray

Thanks for telling your interesting story!

One small item which I would recommend doing differently the next time something like this comes up (you never know icon smile Blocking your Redirect 301 with robots.txt file? Bad Move! ), is not using the robots.txt to block crawling while you have a 301 redirect enabled for the domain. By blocking crawling, you're effectively blocking the search engines from recognizing the redirect properly.

Given the things that you have done so far, I think it's probably not that important to check on the details as shown in the link:-query for your site. Over time, things should normalize — you're on the right path!”

In case if you are concerned as to how long the Redirect 301 should be kept active, then you need not worry, because there is a simpler way. You can always collect as much linkage data as you can from the previous domain. This will enable you to go back to those websites and ask them to update your link to the new domain. You can collect linkage data from Yahoo Site Explorer, Google Webmaster Tools, your web analytics, your own database scripts and more.

4.thumbnail Blocking your Redirect 301 with robots.txt file? Bad Move!

Navneet Kaushal

Navneet Kaushal is the founder and CEO of PageTraffic, an SEO Agency in India with offices in Chicago, Mumbai and London. A leading search strategist, Navneet helps clients maintain an edge in search engines and the online media. Navneet's expertise has established PageTraffic as one of the most awarded and successful search marketing agencies.
4.thumbnail Blocking your Redirect 301 with robots.txt file? Bad Move!
4.thumbnail Blocking your Redirect 301 with robots.txt file? Bad Move!