How to properly loop through these external URLs to get them into the sitemap

  • Resolved Nkululeko

    (@oieniuz)


    8 years ago

    I have filtered Urls that I would like to add to push into a sitemap. I am using one of the sitemap plugins, which has hooks to further modify it.

    my code so far

    // add to theme's functions.php
add_filter('bwp_gxs_external_pages', 'bwp_gxs_external_pages');
function bwp_gxs_external_pages($pages)
{
    return array(
        array('location' => home_url('www.example.com/used-cars/location/new-york/model/bmw'), 'lastmod' => '27/03/2017', 'frequency' => 'auto', 'priority' => '1.0'),
        array('location' => home_url('www.example.com/used-cars/location/los-angeles/model/aston-martin'), 'lastmod' => '27/03/2017', 'frequency' => 'auto', 'priority' => '0.8')
                array('location' => home_url('www.example.com/used-cars/model/mercedes-benz'), 'lastmod' => '27/03/2017', 'frequency' => 'auto', 'priority' => '0.8')
    );
}

    So as you can see in my code that I have these kind of URLs www.example.com/used-cars/location/new-york/model/bmw & www.example.com/used-cars/model/mercedes-benz

    So my issue is that, there are thousands of these URLs and I to push them all into this sitemap.

    So my question is that, isn’t there a way to perhaps loop them over? than to insert them into the code one by one like so

    array('location' => home_url('www.example.com/used-cars/model/aston-martin'), 'lastmod' => '27/03/2017', 'frequency' => 'auto', 'priority' => '0.8')

    • This topic was modified 8 years ago by Nkululeko.
Viewing 5 replies - 46 through 50 (of 50 total)
  • Thread Starter Nkululeko

    (@oieniuz)

    WOOOWW!! IT WORKED!!! You’re a genius! Your knowledge of PHP is insane! I honestly almost gave up on it. I see you also managed to figure out the DB query. I stumbled upon a few glitches in the beginning, but I knew from the get-go that it was caused by one of the plugins I installed, soon as I found which one it was and de-activated it, your plugin worked like a charm.

    What I did was, when I set the hide_empty to false and then I set the Max. Items on 2000 and your plugin split it into 40 sitemap pages as you said it should. This way it outputs 2048 links per sitemap page, so we know for sure that it can handle all the links. This works perfectly fine for me. anything from 8000 Max. Items causes that timeout notice. But overall I’m happy with everything. I still can’t believe you figured it out. I can’t thank you enough. You’re the man!

    Now that we done with the sitemap issue( I still can’t believe we done), It now brings us to that canonical links issue that you brought up. I know what a canonical link is, but I seem not to entirely understand why we need it on these links. Do we still need it even when the hide_empty is true? If the links have no related posts and we place a canonical link on them, does that mean they won’t get indexed by Mr. Google?

    Here’s another puzzle challenge you might find interesting, I tried it. I got stuck, and then I gave up on it(because I was told it is not possible on wordpress), just a quick one. Is it possible to remove taxonomy bases from double term links e.g example.com/used-cars/location/los-angeles/model/audi to example.com/used-cars/los-angeles/audi. If you say yes, I’ll start it as a new topic and post the code where I got stuck and you can tell me what I’m doing wrong. and also, would it make a difference, in terms of page ranking if I removed the taxonomy bases?

    Moderator bcworkz

    (@bcworkz)

    Great news! While I had good reason to expect it to work, we never really know until it goes onto the intended site with real data. My goofy test data only goes so far. I owe much of the success to discovering we can register a sitemap type and Yoast would then handle the rewrites and directing such requests to my callback.

    If each link in the sitemap leads to unique real content, there are no “nothing found” sort of pages, then there is no need to worry about the canonical links, they will be fine as WP handles them by default. It’s only when different links all lead to the same content that search bots have issues with.

    The presence or lack of canonical links does not impact which pages are crawled. It only comes into play when multiple links lead to the same content. Then the bot will check the canonical link. If the canonical link is always the same for a certain given content, even with multiple links leading to it, the bot will see the canonical link is the same and use that for reference and not be concerned with the link used to get there.

    Removing taxonomy bases can sometimes be possible, but doing so often results in complications when WP misinterprets the permalink terms for something else. With bases it’s almost impossible to misinterpret. For example, how would WP know the difference between a taxonomy term and a post slug if there were no taxonomy base saying so? In your /used-cars/los-angeles/audi example, WP will try to find a single used-car post with the slug “audi” while what you really mean is all posts with the term “audi”.

    If you are sure that all permalinks with “used-cars” followed by two more terms will always be location/model requests and never anything else, then something is possible by adding a rewrite rule that matches up with the presence of “used-cars” followed by exactly two more terms. At least I think so. I’m not nearly as good with rewrite rules as I am with generating sitemap links ??

    Thread Starter Nkululeko

    (@oieniuz)

    Alright, I get what you’re saying. So what happens when the hide_empty is true, and lets say we have 10 car posts on this link …/location/new-york/model/bmw right, and all 10 car posts on this link expire in 30 days, so obviously the link will be removed from the sitemap because its empty right, will the link also be removed from Google and then indexed again when the link gets posts?

    Yeah, that would create complications. makes sense. Lets rather leave the tax bases as is then. a link with tax bases and a link without, would get the same ranking on search engines, there isn’t much of a difference right?

    Moderator bcworkz

    (@bcworkz)

    Well, no one (aside from a select few Google engineers) really knows how search providers do things. We believe things work a certain way from observed behavior and deduction. But how they do things keeps changing. SEO “rules” that seemed valid 5 years ago may no longer be valid.

    Eventually Google will catch up with your sitemap state, but it may take a while. I don’t think they pay much attention to the update frequency, which is probably why Yoast does not provide it. If the search bot finds certain things really do change often, it will crawl more often and your changes will be reflected rather quickly.

    If it finds things do not change that often, it will crawl less often, and when things really do change, it can take a while for the change to be reflected on search results.

    Generally speaking, it should work out the way you expect. There will be a certain lag. It may not be very long, or it may take a while. ˉ\_(ツ)_/ˉ

    Thread Starter Nkululeko

    (@oieniuz)

    Alright. Thanks again for helping me out with this. Couldn’t have done it without your help. Much appreciated.

Viewing 5 replies - 46 through 50 (of 50 total)
  • The topic ‘How to properly loop through these external URLs to get them into the sitemap’ is closed to new replies.