Generate URLs no results (with solution)

  • If “Generate URLs” button produces no results, no errors, no warnings, here’s the solution:

    Please add to documentation:

    Solution: un-block (e.g. in .htaccess) the User Agent “PHPCrawl”

    SetEnvIfNoCase User-Agent (PHPCrawl) HTTP_badUserAgent=$1
    # Allow WP Scraper plugin By Allyson Rico, Robert Macchi uses PHPCrawl user agent, for only your specific IP addresses
    SetEnvIfExpr “-R ‘12.34.56.78’ || -R ‘23.45.67.89’ ” !HTTP_badUserAgent
    RewriteRule .* – [L]

Viewing 2 replies - 1 through 2 (of 2 total)
  • Plugin Author Rico Macchi

    (@rico-macchi)

    Thank you for your input.
    I am sure this will help.

    Rico Macchi

    Thread Starter lernerconsulting

    (@lernerconsult)

    Oops, forgot to put those commands inside code tags…

    Those have to be normal double quote and normal single quote (apostrophe) characters, not the fancy word processing quotes.

    Use || (two vertical bar characters, usually shift-backslash on your keyboard) between each IP address, and you can specify as many addresses as you need.

    You specify your actual IP address(es). If your home IP was 98.87.76.65 and your phone IP was 123.45.67.89 and your work IP was 78.89.23.45, the line would be:

    SetEnvIfExpr "-R '98.87.76.65 || -R '123.45.67.89' || '78.89.23.45' " !HTTP_badUserAgent

    So the whole thing would be 

    SetEnvIfNoCase User-Agent (PHPCrawl) HTTP_badUserAgent=$1
# Allow WP Scraper plugin By Allyson Rico, Robert Macchi uses PHPCrawl user agent, for only your specific IP addresses
SetEnvIfExpr "-R '98.87.76.65 || -R '123.45.67.89' || '78.89.23.45' " !HTTP_badUserAgent
RewriteRule .* – [L]

    And of course, copy and paste so you don’t have errors from word wrapping. (The lines start with SetEnvIfNoCase , SetEnvIfExpr and RewriteRule.) The line starting with “# Allow” is a comment.

Viewing 2 replies - 1 through 2 (of 2 total)
  • The topic ‘Generate URLs no results (with solution)’ is closed to new replies.