Panelists explored how you can ensure that large sites with complex databases and lots of content are efficiently crawled by Google. The session covered everything from canonicals and crawl priority to pagination topics.
Moderator:
Alex Bennert (@alexbennert), SEO Consultant, Independent
Q&A Coordinator:
Jason White (@sonray), Director of SEO, DragonSearch
Speakers:
Mark Munroe (@markemunroe), SEO, SEORadar
Anthony Piwarun (@apiwarun), Manager, Search Engine Marketing, Zeon Solutions
Matt Storms (@matt_storms), Independent Consultant, Independent Consultant
Matt Storms
Matt Storms came up with tips for Infinite Scroll and AJAX SEO issues.
Infinite Scroll
It is where you can keep on scrolling and never reach the bottom of the page. When you scroll downward, it would load more content, providing a never-ending amount of material.
He mentioned a few famous sites like Twitter, Facebook, LinkedIn, Chicago Tribute, LA Times, FindNSave, Pinterest and Google Images and asked which one of these was truly an infinite scroll site.
Breaking content into chunks
Take stock of the content that you wish to infinite-scroll and break it up into chunks. Make sure that the users are able to find whatever they are looking for without the scroll going unreasonable. If the user has to keep on the scrolling for long, it will only turn out into disappointment.
Using JavaScript and Ajax
Create a URL structure which accommodates infinite scroll. Avoid two things as you structure the URLs:
- Relative-time-based URL parameters
- Code-based language in the URL
Keeping with best practices
Ensure that the pages have optimal load times and do not duplicate the content of other pages. Each page should include rel=“next” and rel=“prev” within the <head> tag.
PushState Method
It is an HTML5 method which dictates the load process. It will draw from the state object (anything serialized) and the URL to the load the content in sequential manner as determined by the pagination.
PushState directs the browser what to load and display. This is the technique Twitter uses to implement infinite scroll.
ReplaceState Method
ReplaceState method is used for modifying the history entry or responding to user-scrolling behavior. The method would respond to scroll actions and allows the user to view pages which were loaded in the pagination history.
Testing infinite scrolling page
Make sure that every page loads accurately and in the right time frame. Follow it with creating the best and cleanest content possible.
Build better sitemaps
Matt advised to build better sitemaps. He attempted to take home his point with analogy of building blocks.
Track bots crawling your site
He asked the audience who has used New Relic to show hands. Using custom attributes to track Googlebot, Bingbot and others can be effective. Find how the search engines are crawling and what can be done for the change.
Setting up analytics
You can choose from Google Analytics, Adobe Analytics among many others.
Page parameters
You need to monitor your page parameters. Matt explained when Google detects duplicate content such as variations caused by URL parameters, it groups the duplicate URLs into one cluster. It would then select what it determines to be the best URL to represent the cluster in search results.
Properties of the URLs in the cluster, such as link popularity, are consolidated to the representative URL. The process often provides users with more accurate search result.
There are advanced features and you must play with them on your own site before using them on client sites.
Issue with Googlebot Indexing
Matt explained how he dealt with Googlebot indexing issue. He worked on pagination and canonical.
Show as many pages which make sense. For instance,Prev, 1, 2, 3, 4, 5…& Next. Spiders will be able to see the next page with Javascript turned off.
Matt changed the canonical of page #2, #3 and so on.
They did not want to index the Infinite Scroll pages with JavaScript. It will send a signal to the spiders that the page is different, encouraging them to scroll it.
Keeping low quality pages out
To prevent spammy looking duplicate pages from rendering in the indexes, Matt and his team incorporated the noindex, follow tag on each of the /?page=2/pages. They did not have to worry about the page, but more about the items.
Clear signals
These include Canonical, Prev& Next, Noindex, Follow
Load times
You need to ensure the pages have optimal load times and do not duplicate the content. He also asked the audience questions like:
- What is an ideal load page time?
- How many of you (audience) had an Infinite Scroll site?
- Do you like sites that use it?
- Do you feel this is the future of sites?
Bonus Tip
People are unsure whether 301, 404, or 410 is better for product sites. He explained:
301 – old page to new page
302 – should never be used
404 – the page is not found, but search engines will keep coming back to find it
410 – will kill the page within a week
If there are links to the product page, choose 301.
If no links are there, or you do not care, select 410.
Matt concluded by informing that 2.2 billion page were indexed in less than 4 months.
He termed infinite scroll sites the future.
He gave an example of LA Times. You begin reading an article and reach to the bottom, where you find another article similar to what you just read. It is likely to keep you on the site longer.
Mark Munroe
Mark Munroe also provided useful insights into the technical challenges faced by SEOs.
Q&A Session
During his session on technical SEO at SMX, Matt Storms asked attendees, "How many of you consider infinite scroll to be the future of web pages?" But, nobody raised their hands and Storms was surprised. So, a debate began where SMX experts weighed the pros and cons of the infinite scroll, the popular site design.
In favor of Infinite Scroll
The first thought most of the attendees had was the infinite scroll sites are annoying. It was a notion popularized by xkcd. But, it's a layout that appeals to the preferred web behavior of people. Here's the pros aspect covered in the discussion:
The most popular non-commercial websites are doing it, so the average user knows it.
Matt said, "When you use Facebook, you scroll infinitely to see the friends who are doing – till you get bored. This can be up to 30 minutes".
It's mobile friendly
Alex Bennert, a consultant formerly of The Wall Street Journal said, "If we think about how frequently people are searching the web on mobile, it makes sense when they use a hand to literally scroll".
Promotes endless content consumption, especially in the case of ecommerce and news sites
Matt said, "he believes that infinite scroll is the future for news sites. It is a great way of showing endless related content for more time on the website and engagement. The same can be said for ecommerce sites.
Moreover, infinite scroll can help in building KPIs for publishers. Since, it keeps people on the site for longer, the impressions can get a boost.
Can create seamless experience, when done right
Matt said that if the design and the tech integration are solid, people will prefer infinite scroll as the option. It's modern and also where the web is right now, said Storms.
Against Infinite Scroll
Performance issues including the load time
Marcus Tober of Searchmetrics that that a lot of ecommerce websites are turning back to regular designs from infinite scroll, which he says could be because of infinite scroll. Even Storms agreed that it could be an issue. He said, "I admit, the biggest problem with infinite scroll pages is getting the load time under a few seconds".
Navigation nightmare
Mark Munroe of SEORadar said that sometimes people want to look at individual product pages and jump to them quickly.
No inclusion of Google News
"Infinite scroll exceeds the file size for Google News accepted pages", said SEO consultant Alex Bennert.
The Conclusion
As far as user experience is concerned, there's no right answer, unless it comes from the visitor. The core of any design decision should be the customer, else the experience will suffer no matter how great it looks.
