Google has gained an important patent from US Patent office for 'Methods and apparatus for estimating similarity'. Google had filed for it on December 31, 2001. Using this patent, the search engine giant can develop duplicate content detection tools for the webmasters.
The US Patent Office explains on the new patent, “A similarity engine generates compact representations of objects called sketches. Sketches of different objects can be compared to determine the similarity between the two objects. The sketch for an object may be generated by creating a vector corresponding to the object, where each coordinate of the vector is associated with a corresponding weight. The weight associated with each coordinate in the vector is multiplied by a predetermined hashing vector to generate a product vector, and the product vectors are summed. The similarity engine may then generate a compact representation of the object based on the summed product vector”.
A WebmasterWorld Forum thread says,"Anything they do to reduce duplicate content showing up in SERPS is a good thing. Thanks to Google for trying to help web users sick of seeing copies".