English Español Deutsch Français Italiano Português (Brasil) Русский 中文 日本語
Go to Blog
Michael Stricker

Panda 4.0 Detection and Industry Effects

Michael Stricker
Panda 4.0 Detection and Industry Effects

Matt Cutts via Twitter announced the release of the latest incarnation of the Google Panda algorithm update on May 20. Preliminary data shows that Panda 4.0 may have affected 7.5 percent of frequently searched terms in the SERPs, making this an update with potentially far-reaching consequences for many sites.

I intend to contribute a better understanding of the potential effects of the Panda 4.0 algorithm update through data analysis. Early information points to page structure, page types and keywords as being among the potential triggers. This article is an attempt to identify those industries, which may have been hardest hit in order to examine those effects more closely. Based on SEMrush’s data, I believe that some industries appear at first glance to have been either specifically targeted by the algorithm, or they have inadvertently received the brunt of Panda 4.0’s effect.

In this post on the MOZ blog, Dr. Pete highlighted category pages' losses he tracked, like http://www.ebay.com/ bhp / led-fiber-optic-Christmas-tree. In Rishi Lakhani's RefuGeeks blog, his example to illustrate the loss was http://www.ebay.co.uk/ bhp/ iphone-5-case.

Rishi states that eBay’s share of Top 10 Rankings (among MOZcast 10K) fell from #6 to #12, and ultimately tumbled to #25. Some evidence exists that the declining URLs are category or sub-category pages. It appears possible that Google is doing a more stringent job of isolating a single content instance from any given domain to rank well. This treatment of eBay appears to be removing multiple instances of pages from domains and subdomains and directories on the SERPs.

There seems to be an issue where the rank-loss occurred on non-canonical URLs, pages that deviated from strict adherence to directory structure, and are not easily accessible using typical site navigation. Two-thirds of eBay's decimated pages were part of this "phantom" structure, according to SEMrush data.

Such URLs are also treated as subdomain, (popular.ebay.com) pages. This may be rendering those pages as Duplicate Content, the same content as both "popular" subdomain and "/bhp/" directory URLs. The"/bhp/" pages are accessible through the "See also ..." link. Perhaps this has the effect of improving relevance by presenting a string of related keywords, though whether this is punishable is questionable.

Rishil of RefuGeeks pointed out that amped keyword coverage and the anchor footer links on product pages may be transferring domain authority in a deliberate way to page authority. Again, whether that is a contributing factor is a bit cloudy at this time.

On Rishi’s blog post, Grant Simmons commented that the Update is evaluating page quality, duplicate content and utility to users as rank factors. Prior Panda Update losers have correlated strongly with duplicate content.

Some early keyword investigation

Google likes to target highly competitive keyword ranges that elicit hyperactive SEO by commercial interests. We’d love to revisit the keywords in greater depth. The Top 25 Keywords where eBay lost Top 10 rank positions, per Dr. Pete:

  • "Beats by Dr. Dre"
  • "Honeywell thermostat"
  • "Hooked on Phonics"
  • "fajate"
  • "Batman costume"
  • "Lenovo tablet"
  • "Pyramid Collection"
  • "Hampton Bay"
  • "Jordan 11 Concord"
  • "pontoon boats for sale"
  • "Mockingjay pin"
  • "Kobe vii"
  • "food trucks for sale"
  • "Galaxy s2"
  • "Jordan Spike"
  • "foamposite"
  • "George Foreman Grill"
  • "wholesale jerseys"
  • "tend skin"
  • "Fender Stratocaster"
  • "rims for sale"
  • "shed plans"
  • "Hello Kitty Vans"
  • "cheap used cars"
  • "Lilly Pulitzer bedding"

Many of these key phrases include brand and unique product names. Thus, if one were a user searching for a specific product, then Google might (rightfully) presume that you would be better served by finding the product page, and not the category or sub-category page.

E-commerce might tend to put lots of alternatives in front of shoppers on category pages and popular pages in a bid to gain conversion and to prevent bounces and loss of revenue. By presenting slimmer, more-focused product pages in response to product queries, Google may be placing consumers closer to a conversion. That shorter user journey may translate to the "searcher satisfaction" Google yearns for.

Big Losers (Searchmetrics) include:

  • Ask.com
  • Ebay.com
  • Biography.com
  • Retailmenot.com
  • Starpulse.com
  • History.com
  • Isitdownrightnow.com
  • Aceshowbiz.com
  • Examiner.com
  • Yellowpages.com
  • Yourtango.com
  • Dealcatcher.com
  • Livescience.com
  • Webopedia.com (one of the ancient internet.com domains)
  • Xmarks.com (social bookmarking as a browser toolbar service)
  • Simplyrecipes.com
  • Siteslike.com
  • Digitaltrends.com
  • Health.com
  • Spoonful.com

Big Winners include:

  • Glassdoor.com
  • Emedicinehealth.com
  • Medterms.com
  • Yourdictionary.com
  • Shopstyle.com
  • Zimbio.com
  • Myrecipes.com
  • Couponcabin.com
  • Buzzfeed.com
  • Consumeraffairs.com
  • Wordpress.com
  • Thinkexist.com
  • Onhealth.com
  • Alternativeto.net
  • Whosdatedwho.com
  • Reverso.net
  • Wikimedia.org
  • Dogtime.com
  • Findthebest.com
  • Eatingwell.com

From those short lists, I’ve extracted a more focused list of those industries which I believe have been targeted. These industries may also be subject to their own over-optimization tendencies due to degree of competition.

High-SERP-Flux Industries

Searchmetrics' Marcus Tober singles out Losers as sites focused on:

  • Health & Nutrition
  • Search Engines (perhaps with massively-interconnected but slim content)
  • Coupons
  • News (especially the celebrity/ gossip sector),
  • Software
  • Price Comparison (Deals and Coupons)
  • Health Pages

Panda Update 4.0 Testing by Industry Leaders

Below I have tested five of the most popular, highly trafficked domains in each industry category. One reason is to detect whether particular industries might have been singled out. Readers may also survey the winners, losers and static domains below, to extract their own conclusions about what factors Panda 4.0 may be focused on. I welcome your comments.

Top Five Online Retailers (from Netonomy.net)

Amazon.com - disproportionate, but not extreme, losses in a pretty stable environment


Staples.com – a fair winner in a volatile space


Apple.com - largely unaffected


Walmart.com - more winners than losers on a roller-coaster ride of new and lost keywords


Dell.com - not much effect


Top Five Health Domains (from Ebizmba)


webmd.com – a bit of winning, yet considerable flux, on this extraordinarily stable domain with very little emergence and departure of keywords, otherwise


NIH nih.gov – This highly trusted .gov domain fairly mimics the previous profile with regard to low turnover, though the scale is twice that of WebMD


Yahoo! Health

health.yahoo.com – here is some serious loss that has been consistently applied for the first few days of Panda 4.0 Update. Yahoo loves to present popular items using multiple paths – could this be a “duplicate content” factor?



mayoclinic.com – big, big losses here for pages that either are, or should be, redirected. Very little chance that pages from the .com TLD will be indexed or presented in SERPs in the future.


MayoClinic (primary domain)

mayoclinic.org – some gains, perhaps gained from the losses on their .com side



medicinenet.com – some loss taking place on this otherwise stable domain


Nutrition and diet domains might be a good subject for future examination.

Top Eight Health & Medicine Domains (from Refseek)

Some deeper examination into this particular industry seemed warranted by the amount of Panda 4.0-related activity. This provides a broad overview across a span of TLDs and across business, non-profit, community and government models.

American Cancer Society

cancer.org – increased, but balanced activity



CDC.gov – increased activity with a net downside


Drugs.com - some fairly serious downside, proportionally similar on each subsequent day of the Panda 4.0 application


Department of Health and Human Services

hhs.gov – prior low and balanced entry and exit of keywords is slightly amplified and still fairly balanced


EverydayHealth.com - big winner! Where are these keyword shares being taken from? Could it be Hardin? iVillage?


Hardin Health

hardinmd.lib.uiowa.edu - sizeable losses based on content may be surprising for an .edu TLD



health.ivillage.com - Looks to be the biggest loser among popular domains in this industry. The scale is not enormous, yet the result is a stinging indictment of this community’s methods


Intelihealth – again, not a huge volume of keywords, yet a huge winner, proportionately


Top Five Coupon Domains (from Ebizmba)

While some industries have apparently come under increased Google scrutiny and processing of keywords into and out of the SERPs, this particular vertical shows evidence of overall similar levels of activity, both before and after Panda 4.0.

Groupon – Amazingly, this runs counter to the experience of most domains surveyed. We see a resumption of keywords entering and leaving the SERPs at a reduced level of flux.


RetailMeNot – one of Searchmetrics’ biggest losers, proportionately, yet not the largest volume of affected keywords. Of course, keyword query volumes must be a consideration.


ShopAtHome – big winner! What are they doing well?


Coupons – quite stable, and even quieter than before the update


SlickDeals.net – a slight downturn in a fairly low volume of keyword turnover


Top Five Celebrity Gossip Domains (from Ebizmba)

The celebrity gossip media industry as a whole may be subject to an entirely new level of “freshness dating” that is intended to surface the latest news items for searchers. This may have a detrimental effect on long-term visibility of popular items that have built up strength over time.

Yahoo! OMG! Omg.yahoo.com – the activity levels are obviously amplified and growing, and the overall effect appears fairly balanced, with some downside


WonderWall wonderwall.msn.com – some greater activity, some overall and mounting losses


TMZ.com – this gossip site exhibits the increased flux of the industry as a whole, with balanced overall effect.


EOnline.com – a profile similar to others in the Industry


ZimBio.com – Panda 4.0 has been very kind to ZimBio, with inordinately higher attention and great gains


WhosDatedWho - another Big Winner – after some prior losses almost a month ago


Top Five Software Download Domains (from Toptenreviews)

One might expect that a volatile industry with high-onset rates of new products and large volumes of abandoned apps might be subject to higher-frequency examination and lots of flux. Freshness of data would likely be a quality factor. Let’s see whether that’s the case for some of the most popular domains.

Download.com - big loser proportionately, until one considers that the activity is taking place on a small number of keywords


BrotherSoft.com - big winner, and that win applies to a high number of keywords. The gains appear likely to continue, given the increased attention that they are getting. This signals significant entries into the SERP ranks 20 and above.


Tucows.com - much more flux, fairly flat results


Softpedia.com - big proportional losses at high volume – ouch


Softonic.com - fair gains at very high volume – a big winner emerges from a high level of SERP activity



30-day periods are examined. This data derives from a sample of search terms. Your proprietary analytics data about search-driven traffic from Google may differ.

Synchronous rollout of “Payday – Loans 2.0″ also occurred during this same timeframe. This, like its predecessor, affected the highly spammed “money terms” that feed the personal finance sector. This filter is more likely to result in de-indexations as a penalty, rather than rank losses. This makes it easier to identify the cause if one is targeted.

Thanks to Searchmetrics’ Winners and Losers for much of the data I used to start my research.

Preliminary Conclusion

It is essential that you assess whether or not the Panda 4.0 Update has affected your site. 7.5% of SERPs may have been affected, but some of those domains (eBay) have shown up to 65% of their URLs affected as well. Assessment is easy, especially if you use the Organic Keyword Report in SEMrush.

If you have been affected, see your Organic Keyword Competitors Report (or use your own knowledge of who your SERP competitors are), to examine how they fared. It is clear that in many industries, there have been winners and losers. Assess who may have gained share, and who has lost. Then, examine the design and content of those pages most affected to decipher the differences and discover any applicable edits to improve your own results.

Some are pointing to the “duplicate content” factor as a culprit, so pay attention to the URL structure and be mindful of subdomains, intervening directories and canonicalization. Google may be doing a more stringent job of indexing and displaying a single page per domain for product keywords. It would be a pity to be victimized by your own canonicalization when you might prefer to have other pages rank highly.

SEMrush provides the ultimate in competitive intelligence for your business. Analyze our unparalleled collection of historical data to identify trends and form the perfect digital marketing strategy for your organization. Email SEMrush (mail at semrush dot com) to learn more about how the tool can change the way you think about competitive analysis, as well as pricing and plans.

A special thanks to Barry Schwartz at Search Engine Land for his research into this issue!

Offer your own insights in the comments below, and stay tuned to our blog for voices from the SEO world as they comment on Panda’s effect on the industry.

Michael Stricker

Provides valuable insights and adds depth to the conversation.

Michael Stricker is a former SEMrush Director of Marketing. His industry musings can be found on Twitter and on our blog. His last post was "What are the Top SEO Issues Impacting your Site?"
Send feedback
Your feedback must contain at least 3 words (10 characters).

We will only use this email to respond to you on your feedback. Privacy Policy

Thank you for your feedback!