According to Matt Cutts, small, medium as well as big websites should be very careful about XSS holes on their websites. These XSS holes can vary in forms, but one of the most common forms is the 'search boxes'. Clicking on these 'search boxes' is a big NO! NO!.
In the event, that you may notice a dip in your website's Google rankings, then you should initiate a few searches that would help you to find, if anyone has injected spammy or porn content on your website.
For example: If your website is example.com, you can run a few queries such as [site:example.com porn] or [site:example.com biaxin] or [site:example.com viagra] to see whether you run across unexpected results.
There are also a couple of blogs at Google Online Security Blog and Google Webmaster Central Blog, that highlight and address this issue.
Note: (Switching from XSS to pure hacked sites for a moment.) Make sure to change your admin password if you update (say) your WordPress installation. Sometimes hackers are smart enough to save your password and come back even after you've fully patched your system.