Google Crawling WF URLs

  • Resolved captaincrank

    (@captaincrank)


    2 years, 6 months ago

    Google has begun crawling an excessive number of WF URLs. The URLs Google is crawling fit this format:
    domain.com/?wordfence_lh=1&hid=708DA0E8D3154F40C9D0BEE9E08DF3E3

    In Search Console, I see on 8/1/2022 49 WF URLs were crawled. By 9/2/2022 the number of WF URLs crawled has ballooned to 924 URLs. What’s the best way to deal with this and stop its impact on our crawl budget? Google killed the URL parameter tool in GSC, so that’s not an option.

    I see the WF pages are indeed blank, and there’s nothing in the code regarding noindex, etc. Code from blank page below:

    <html><head><link rel="stylesheet" href="resource://content-accessible/plaintext.css"><style type="text/css">.c1-ui-state-hover {background-color:yellow !important; padding:5px !important }</style></head><body><pre></pre></body></html>

Viewing 1 replies (of 1 total)
  • Plugin Support wfpeter

    (@wfpeter)

    Hi @captaincrank,

    You’re right to check whether the pages are blank when viewed, as that’s the intended outcome. For this reason, Google doesn’t usually index them as every site with Wordfence installed will use URLs that look like this due to their requirement by Live Traffic. Do you, or have you ever had a theme/plugin where visitors are redirected to the homepage in the event of a 404? Very often, Google will end up indexing these pages when it shouldn’t because of this reason.

    If you’ve tried clearing any server caches or caching plugins you may have and this behavior continues, can you send a diagnostic report to wftest @ wordfence . com? You can find the link to do so at the top of the Wordfence > Tools > Diagnostics page. Then click on “Send Report by Email”. Please add your forum username where indicated and respond here after you have sent it.

    NOTE: It should look as follows – Screenshot of Tools > Diagnostic > Send by Email

    Thanks,

    Peter.

Viewing 1 replies (of 1 total)
  • The topic ‘Google Crawling WF URLs’ is closed to new replies.