For those curious to know how Google is combating spam, there’s a website “How Search Work”. Matt Cutts has released a video on How Search Works website, letting people know the interesting areas within the site they should be navigating to as well as what all Google has been doing to improve the search.
In the video, Cutts says that How Search Works is a small site that talks about advanced steps Google has taken in crawling, algorithms, and how it fights spam.
Even if you have seen the site before, Cutts wants you to walk you through some of the things that you might not have seen and are quiet nice. So, here are the details of what Cutts has revealed about the site in the video.
If you go to the main page of the site and scroll through a little bit, you will find almost an infographic, which is actually interactive. You can click around and find all kinds of fun easter eggs. If you get down the bottom of the page, it will tell you that you have been on the page for 150 seconds. In that time Google has actually handled 5.7 million searches or something along those lines.
As you look through the site, Matt says that you will find Google talking about how it does evaluation. There are videos on how Google evaluates search quality. Google do evaluate new search algorithms and send them out to quality raters and they look at one set of results on left and one set of results on right. They have to decide which one looks better. They don't know what algorithms have been evaluated. And when they vote, Google takes the results and figure out which search results got better and which ones got worse. But, Google doesn't directly apply those ratings into ranking algorithm.
Google actually shows the funnel for the things in recent year. In 2012, Google went through 118,000 ideas where it played around with new way of generating search results and using the ratings it got from quality raters, it was able to say that this looks like a promising experiment. From there, it did 10,000 side by sides, where you get side-by-side set of search results and ask people which one you like better. Based on that, Google did 7000 live traffic experiment where it actually take an experiment and put it on the main website and look how often people click at actual search results to try to determine it was actually making the search results better. Matt also said that as a result, Google was able to 665 algorithmic changes on search result page in 2012. If Google puts that into context, it's roughly two changes to how it generates search engine pages every single day for the entire year.
People come and ask what happened on such and such day because there's lot of stuff happening because there's usually lot of stuff happening. Things rolling out, new data being deployed, and those are actual changes and not the data being refreshed. So, it gives you a little bit of feel for the scale of how many different changes were exploring at any given point.
The part of How Search Works, Matt Cutts enjoys the most is the spam section. Google goes into all kind of information, you might not have seen before. For example, there's a spam carousel and that is updated periodically. So, you actually get to see spam right after Google has removed it. A screenshot is showed so that you don't get infected by malware or something. It's literally like you can watch over Google's shoulder as it is removing spam. So, you get a chance to see the sort of stuff Google has to deal with every single day.
Below the spam carousel, the site has different types of spam; the categories of spam -cloaking or sneaky redirects, hacked sites, keyword stuffing, hidden text, pure spam, and parked domains. Others include spamming free hosts or dynamic DNS providers, thin content with little added value, unnatural links from a site, unnatural links to a site, user-generated spam. There are more specific, more granular, and more detailed things within each one of those. So, unnatural links from a site might involve someone who is selling links that pass page rank. But, that gives an idea of the overall categories Google look at whenever, it is fighting spam.
Google also gives different graphs to tell month by month the actions taken of taken on spam. The vast majority of what Google tackles is what it classifies as pure spam or black hat spam. It means, the gibberish stuff that anybody would be able to recognize if they are sufficiently savvy.
Another thing that you would notice is the next biggest category within recent years has been hacked site. In 2010, some SEOs wrote "what has the web spam team been doing, we haven't seen a lot of action from them recently". Google was at that time engaged in a hand-to-hand combat on hacked sites, which a regular SEO or a black hat SEO might not have noticed as much.
You can get these kinds of insights when you look through these graphs and the history of the sort of stuff that Google has had to tackle in terms of spam.
Google has also started to do more messaging over time. It can do probably better to get more concrete and actionable messages to webmasters. If you look at the milestones of what Google has done, it is pretty exciting as you can see volume spike up as Google has started to get more and more information. At this point, for pretty much any direct action that you take, that the manual webspam team takes that affects you ranking, the webmaster will get a message about that. That's really helpful because you know there's an issue and you can start to deal with it and dig into it and start to investigate a little bit.
In January 2013, Google sent over 431,000 messages as a result of actions that it took on the webspam team. So the other thing you should think about is the scale at which Google is operating. These are all manual webspam actions, which ten generated some sort of message to the webmaster. The idea that Google can have one-on-one conversation with 431,000 different owners of websites sort of shows the scale that Google is operating at and why it's hard and why so far it hasn't figured out a way to have a one-on-one conversation with every single webmaster who wants to rank number one or rank highly or has questions about potential webspam action.
Below this information, is a graph that shows the reconsideration requests that have been submitted. And so for a random week in 2013, there were roughly 5,000 reconsideration requests and over a month 430,000 messages go out. If you turn this week long base line into a month, about 20,000 reconsideration request processing messages is what Google handles during a month. What is interesting is that out of all the people that Google sends messages about manual webspam action being taken, only 5 percent request reconsideration. So, most of the time Google is killing spam, spammers decide to move on to try to do it on a different URL. It's kind of neat to take some of these numbers, compare them, and play a little bit with realizing what insights we can get from these kinds of graphs. It shows you the scale of the problem. If you have 20,000 people a month who want to talk to you why they think their website should rank highly when Google thinks that it has at least violated the guidelines, you see the sort of difficulties Google has in trying to talk to everybody. Google will keep trying to do better, be more transparent, but it's fantastic that Google has got How Search Works website and some dashboards where you can see how things are going. You can see live examples of spam, as they get thrown out. Cutts thinks that you will really enjoy the website.How Search Works: An Interesting Website on How Google is Improving Search Results!,