Duplicate Content Issue

  • Resolved vahost

    (@vahost)


    9 years, 3 months ago

    The directory creates a major duplicate content issue in Google, which leads to any website using the directory plugin to potentially lose rankings in the Google search results.

    Google began penalizing sites with duplicate content in January 2011 algorithmically with the Panda update.

    Steps to recreate the problem:

    (1) Let’s assume you have a category in your directory called Photography And Video, and it has over 200 entries in it. You initially decide to allow 10 listings per category page. So you visit Directory Admin > Manage Options > set Listings Per Page and set the value to 10.

    (2) Google spiders your pages and adds 20 (200 listings at 10 per page) category pages under the Photography And Video category.

    (3) You later decide that you don’t want all those category pages listed on separate pages. You’d rather show the entire category on one page. So you go once again to Directory Admin > Manage Options > set Listings Per Page and change the value to 0. This puts all listings on the same category page.

    (4) You visit any category page, such as:
    https://mydomain.tld/business-directory/my_category/photography-and-video/

    Then you right-click on the page and (depending on which browser you use) click View Source. You’ll find a line in the HTML code that says something like this:

    <link rel=”canonical” href=”/business-directory/my_category/photography-and-video/” />

    (5) Here’s where the problem kicks in. You then visit a numbered page under the same category, such as:
    https://mydomain.tld/business-directory/my_category/photography-and-video/page/12/ and you’ll see exactly the same page in every detail!

    Choose View Source again. This time you’ll find a line that says something like this:

    <link rel=”canonical” href=”/business-directory/my_category/photography-and-video/page/12/” />

    The problem here is: that page shouldn’t exist! Once I changed the value of Directory Admin > Manage Options > set Listings Per Page to 0, that page should disappear entirely from the directory structure!

    But since it DOES exist and DOES show an actual page, Google can (and will) spider it and continue to index the page, creating a duplicate content condition.

    Important note: both category pages contain the exact same content … all of the listings in the Photography And Video category. Both pages claim to be the canonical URL for that page’s content.

    In our little scenario, there are now 19 additional pages in the Google index that will never disappear, because any page value you enter into the URL will show the same category page.

    Additionally, there are a bunch of other duplicate pages for all of the other categories in the directory.

    Google then penalizes your website for duplicate content on a massive scale. Worse yet, unless you’re aware of how this kind of thing works, you’ll never know about it, and you’ll never understand why you can’t get any free search traffic to your website on a regular, ongoing basis in any significant numbers.

    https://www.ads-software.com/plugins/business-directory-plugin/

Viewing 4 replies - 1 through 4 (of 4 total)
  • Plugin Author Business Directory Plugin

    (@businessdirectoryplugin)

    Thanks for pointing this out, vahost. We definitely weren’t aware of that and we’d like to fix something like that because as you noted, it’s pretty serious from Google’s perspective.

    We’ll check into it right away and if we can’t reproduce it, we’ll need to contact you to see the site you’re talking about. If you can post it, that may help–or contact us here to send it along with a link to this forum thread: https://businessdirectoryplugin.com/contact/

    Plugin Author Business Directory Plugin

    (@businessdirectoryplugin)

    We did some deep investigation here–turns out this is a core WordPress bug that is causing this issue:

    This is the WP bug:

    https://core.trac.www.ads-software.com/ticket/28081
    https://core.trac.www.ads-software.com/ticket/11694

    And it’s not exclusive to us, either. It affects a lot of things–even regular WP pages. This is an example from our own site that is a non-BD page: https://businessdirectoryplugin.com/features/ and https://businessdirectoryplugin.com/features/page/10/.

    We have added a workaround in our directory to handle this for 3.6.8, but that will only address the issue with BD pages and content. The bug will still exist on sites because of WordPress, unfortunately.

    Thanks again for bringing this to our attention.

    Thread Starter vahost

    (@vahost)

    The 3.6.8 workaround does indeed remove the listings from the pages. However, the pages themselves still exist. They don’t generate a 404 page not found error, and they don’t have a “noindex” meta-tag, so the search engines still believe they should spider those pages. Instead, the main page still generates a 200 page found result code, telling Google that the page should still be spidered and is still a legitimate page.

    You’ve included a canonical tag that says that the canonical url is the home page of the site. A better practice is to make the main category page’s url the canonical url.

    If you can’t return a 404 page not found error instead of showing a page with no listings on it, at least build a rel=”noindex” meta-tag into those otherwise “blank” pages, so that Google will know to remove the page from its index.

    Plugin Author Business Directory Plugin

    (@businessdirectoryplugin)

    Hi vahost,

    We no longer display the duplicated listings, but we can’t make the page disappear. By the time our code is called (during the page loading process), the time to return a 404 error has passed. We can’t work around WP itself.

    404 errors are sent via a header so they should be sent before content is created. Meta tags are added between <head> tags, but our code runs when WP is processing the shortcode (which happens too late).

    We can only figure out that the query is not going to return listings when it is too late to send a 404 error or modify the meta tags. There’s no easy way around this until WP either fixes the problem or we stop using the short code method here. As much as I’d like to run the queries outside of the shortcode code, that would require an extensive modification to both core and the premium modules.

    I would strongly suggest going to the issues above and voicing your opinion about the WP core not doing what it should and the consequences of it on a WP site in general. Changing our stuff still only addresses a part of the real issue here.

Viewing 4 replies - 1 through 4 (of 4 total)
  • The topic ‘Duplicate Content Issue’ is closed to new replies.