Digitalghost has started a thread at threadwatch about Google's Most Common Penalty.
There are generally two Google penalties which people are aware of the Google +-30 penalty and the 950 penalty. However according to digitalghost there is a third penalty which is unofficially called 'My Site Sucks Ass And Google Just Figured It Out'. People do not talk much about it as it requires that the person be held responsible.
It's quite simple really. Google gives webmasters a chance. Also known as 'give them enough rope'. Sites start out with an idea and a hardworking webmaster. But somewhere along the way, the content ends up being a remash of all the content that has gone before.
Ads take over content real estate. So A Widget Forum is added. And a Widget Resource page, and a Widget Directory, and soon, the site is just a replica of all the other widget sites. With fewer links and a bored webmaster. In short, the site sucks ass. But there for awhile, Google couldn't tell that it sucked ass. So they waited. (see, Give Them Enough Rope above).
Unfortunately, the benevolent Google god's patience inspires hope in the poor Webmaster. Google loves me! This is easy! Then, when the wrath of the Google Gods is unleashed upon the poor faithful, they, like humans have for centuries, blame everyone but themselves.
This pretty much explains why some of the sites are not recognised by Google even after a lot of efforts have been blindly put into it.