Google is observing this behavior on many video sites, mostly adult and has come up with a strict note for these site owners in the recent Google Webmaster Central Blog. Such websites display content provided on an affiliate program and the same content is available across several hundreds or thousands of other websites also.
In case your website also syndicates the content, which is also available elsewhere, you have to figure out "Is the site providing significant additional benefit so that the user would like to visit this site again in search results, instead of going to the original source of the content"?
If the answer is No, there are chances that the website might frustrate the searchers & violate Google's quality guidelines. In the blog post, Google has clearly mentioned that it might take action such as removal of the site from the index, to maintain the quality of search results. For any questions related to Google guidelines, you can refer to Webmaster Help Forum.Google to be Strict on Sites Running Affiliate Program without Delivering Valuable Content!,