Unknown URLs not producing 404s
-
I have a very small WP site with too many visitors who look for a URL similar to /wiki/anything/… which does not exist. In fact the /wiki folder does not exist yet 404s are not produced. Can anybody help me get rid of these resource hogs? TIA
-
Please provide a link to a page on your site where we can see this. Thanks.
Hi Steve,
Thanks for the response. I should have noted that there is no “page” but here is what I seemore detail. My small website is https://www.thesturdytree.com. The home page tells “Who’s Online” and usually displays a count of 30-50 guests. I am using a plug-in “Visitor Maps and Who’s Online” that gives members a dynamic page of those guests. The requested URL is always /wiki/something…. A screenshot is attached. HTH but I can provide whatever is needed.
Some background: Several months I installed but never used or configured a wiki that installed into the /wiki directory. I got busy, etc. and didn’t look at the site (I thought wiki was inop) for about three months. When I did look, I discovered hundreds/thousands of URLs attempting to logon or edit wiki data (mostly advertising but all unwanted. I immediately deleted all known files in the wiki folder but they continue. No 404s have been produced (strange?).
Again, thanks
Please show your .htaccess file.
Sorry about that attachment. I see that doesn’t work here. Here is the .htaccess
snip
# BEGIN iThemes Security – Do not modify or remove this line
# iThemes Security Config Details: 2
# Enable HackRepair.com’s blacklist feature – Security > Settings > Banned Users > Default Blacklist
# Start HackRepair.com Blacklist
RewriteEngine on
# Start Abuse Agent Blocking
RewriteCond %{HTTP_USER_AGENT} “^Mozilla.*Indy” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^Mozilla.*NEWT” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^$” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^Maxthon$” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^SeaMonkey$” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^Acunetix” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^binlar” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^BlackWidow” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^Bolt 0” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^BOT for JCE” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^Bot mailto\:craftbot@yahoo\.com” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^casper” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^checkprivacy” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^ChinaClaw” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^clshttp” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^cmsworldmap” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^Custo” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^Default Browser 0” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^diavol” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^DIIbot” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^DISCo” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^dotbot” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^Download Demon” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^eCatch” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^EirGrabber” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^EmailCollector” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^EmailSiphon” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^EmailWolf” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^Express WebPictures” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^extract” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^ExtractorPro” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^EyeNetIE” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^feedfinder” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^FHscan” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^FlashGet” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^flicky” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^g00g1e” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^GetRight” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^GetWeb\!” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^Go\!Zilla” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^Go\-Ahead\-Got\-It” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^grab” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^GrabNet” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^Grafula” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^harvest” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^HMView” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^Image Stripper” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^Image Sucker” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^InterGET” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^Internet Ninja” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^InternetSeer\.com” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^jakarta” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^Java” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^JetCar” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^JOC Web Spider” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^kanagawa” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^kmccrew” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^larbin” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^LeechFTP” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^libwww” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^Mass Downloader” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^microsoft\.url” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^MIDown tool” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^miner” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^Mister PiX” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^MSFrontPage” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^Navroad” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^NearSite” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^Net Vampire” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^NetAnts” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^NetSpider” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^NetZIP” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^nutch” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^Octopus” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^Offline Explorer” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^Offline Navigator” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^PageGrabber” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^Papa Foto” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^pavuk” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^pcBrowser” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^PeoplePal” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^planetwork” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^psbot” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^purebot” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^pycurl” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^RealDownload” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^ReGet” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^Rippers 0” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^sitecheck\.internetseer\.com” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^SiteSnagger” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^skygrid” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^SmartDownload” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^sucker” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^SuperBot” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^SuperHTTP” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^Surfbot” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^tAkeOut” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^Teleport Pro” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^Toata dragostea mea pentru diavola” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^turnit” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^vikspider” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^VoidEYE” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^Web Image Collector” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^WebAuto” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^WebBandit” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^WebCopier” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^WebFetch” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^WebGo IS” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^WebLeacher” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^WebReaper” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^WebSauger” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^Website eXtractor” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^Website Quester” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^WebStripper” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^WebWhacker” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^WebZIP” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^Widow” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^WPScan” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^WWW\-Mechanize” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^WWWOFFLE” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^Xaldon WebSpider” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^Zeus” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “^zmeu” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “360Spider” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “CazoodleBot” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “discobot” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “EasouSpider” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “ecxi” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “GT\:\:WWW” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “heritrix” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “HTTP\:\:Lite” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “HTTrack” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “ia_archiver” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “id\-search” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “IDBot” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “Indy Library” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “IRLbot” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “ISC Systems iRc Search 2\.1” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “LinksCrawler” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “LinksManager\.com_bot” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “linkwalker” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “lwp\-trivial” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “MFC_Tear_Sample” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “Microsoft URL Control” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “Missigua Locator” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “MJ12bot” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “panscient\.com” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “PECL\:\:HTTP” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “PHPCrawl” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “PleaseCrawl” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “SBIder” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “SearchmetricsBot” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “SeznamBot” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “Snoopy” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “Steeler” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “URI\:\:Fetch” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “urllib” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “Web Sucker” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “webalta” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “WebCollage” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “Wells Search II” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “WEP Search” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “XoviBot” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “YisouSpider” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “zermelo” [NC,OR]
RewriteCond %{HTTP_USER_AGENT} “ZyBorg” [NC,OR]
# End Abuse Agent Blocking
# Start Abuse HTTP Referrer Blocking
RewriteCond %{HTTP_REFERER} “^https?://(?:[^/]+\.)?semalt\.com” [NC,OR]
RewriteCond %{HTTP_REFERER} “^https?://(?:[^/]+\.)?kambasoft\.com” [NC,OR]
RewriteCond %{HTTP_REFERER} “^https?://(?:[^/]+\.)?savetubevideo\.com” [NC]
# End Abuse HTTP Referrer Blocking
RewriteRule ^.* – [F,L]
# End HackRepair.com Blacklist, https://pastebin.com/u/hackrepair# Ban Hosts – Security > Settings > Banned Users
SetEnvIF REMOTE_ADDR “^95\.110\.200\.241$” DenyAccess
SetEnvIF X-FORWARDED-FOR “^95\.110\.200\.241$” DenyAccess
SetEnvIF X-CLUSTER-CLIENT-IP “^95\.110\.200\.241$” DenyAccess<IfModule mod_authz_core.c>
<RequireAll>
Require all granted
Require not env DenyAccess
Require not ip 95.110.200.241
</RequireAll>
</IfModule>
<IfModule !mod_authz_core.c>
Order allow,deny
Allow from all
Deny from env=DenyAccess
Deny from 95.110.200.241
</IfModule><IfModule mod_rewrite.c>
RewriteEngine On# Reduce Comment Spam – Security > Settings > WordPress Tweaks > Comment Spam
RewriteCond %{REQUEST_METHOD} POST
RewriteCond %{REQUEST_URI} /wp-comments-post\.php$
RewriteCond %{HTTP_USER_AGENT} ^$ [OR]
RewriteCond %{HTTP_REFERER} !^https?://(([^/]+\.)?thesturdytree\.com|jetpack\.wordpress\.com/jetpack-comment)(/|$) [NC]
RewriteRule ^.* – [F]
</IfModule># Protect System Files – Security > Settings > System Tweaks > System Files
<files .htaccess>
<IfModule mod_authz_core.c>
Require all denied
</IfModule>
<IfModule !mod_authz_core.c>
Order allow,deny
Deny from all
</IfModule>
</files>
<files readme.html>
<IfModule mod_authz_core.c>
Require all denied
</IfModule>
<IfModule !mod_authz_core.c>
Order allow,deny
Deny from all
</IfModule>
</files>
<files readme.txt>
<IfModule mod_authz_core.c>
Require all denied
</IfModule>
<IfModule !mod_authz_core.c>
Order allow,deny
Deny from all
</IfModule>
</files>
<files wp-config.php>
<IfModule mod_authz_core.c>
Require all denied
</IfModule>
<IfModule !mod_authz_core.c>
Order allow,deny
Deny from all
</IfModule>
</files># Disable Directory Browsing – Security > Settings > System Tweaks > Directory Browsing
Options -Indexes<IfModule mod_rewrite.c>
RewriteEngine On# Protect System Files – Security > Settings > System Tweaks > System Files
RewriteRule ^wp-admin/install\.php$ – [F]
RewriteRule ^wp-admin/includes/ – [F]
RewriteRule !^wp-includes/ – [S=3]
RewriteRule ^wp-includes/[^/]+\.php$ – [F]
RewriteRule ^wp-includes/js/tinymce/langs/.+\.php – [F]
RewriteRule ^wp-includes/theme-compat/ – [F]# Disable PHP in Uploads – Security > Settings > System Tweaks > PHP in Uploads
RewriteRule ^wp/wp\-content/uploads/.*\.(?:php[1-7]?|pht|phtml?|phps)$ – [NC,F]# Disable PHP in Plugins – Security > Settings > System Tweaks > PHP in Plugins
RewriteRule ^wp/wp\-content/plugins/.*\.(?:php[1-7]?|pht|phtml?|phps)$ – [NC,F]# Disable PHP in Themes – Security > Settings > System Tweaks > PHP in Themes
RewriteRule ^wp/wp\-content/themes/.*\.(?:php[1-7]?|pht|phtml?|phps)$ – [NC,F]# Filter Request Methods – Security > Settings > System Tweaks > Request Methods
RewriteCond %{REQUEST_METHOD} ^(TRACE|DELETE|TRACK) [NC]
RewriteRule ^.* – [F]# Filter Suspicious Query Strings in the URL – Security > Settings > System Tweaks > Suspicious Query Strings
RewriteCond %{QUERY_STRING} \.\.\/ [OR]
RewriteCond %{QUERY_STRING} \.(bash|git|hg|log|svn|swp|cvs) [NC,OR]
RewriteCond %{QUERY_STRING} etc/passwd [NC,OR]
RewriteCond %{QUERY_STRING} boot\.ini [NC,OR]
RewriteCond %{QUERY_STRING} ftp: [NC,OR]
RewriteCond %{QUERY_STRING} https?: [NC,OR]
RewriteCond %{QUERY_STRING} (<|%3C)script(>|%3E) [NC,OR]
RewriteCond %{QUERY_STRING} mosConfig_[a-zA-Z_]{1,21}(=|%3D) [NC,OR]
RewriteCond %{QUERY_STRING} base64_decode\( [NC,OR]
RewriteCond %{QUERY_STRING} %24&x [NC,OR]
RewriteCond %{QUERY_STRING} 127\.0 [NC,OR]
RewriteCond %{QUERY_STRING} (globals|encode|localhost|loopback) [NC,OR]
RewriteCond %{QUERY_STRING} (request|concat|insert|union|declare) [NC,OR]
RewriteCond %{QUERY_STRING} %[01][0-9A-F] [NC]
RewriteCond %{QUERY_STRING} !^loggedout=true
RewriteCond %{QUERY_STRING} !^action=jetpack-sso
RewriteCond %{QUERY_STRING} !^action=rp
RewriteCond %{HTTP_COOKIE} !wordpress_logged_in_
RewriteCond %{HTTP_REFERER} !^https://maps\.googleapis\.com
RewriteRule ^.* – [F]# Filter Non-English Characters – Security > Settings > System Tweaks > Non-English Characters
RewriteCond %{QUERY_STRING} %[A-F][0-9A-F] [NC]
RewriteRule ^.* – [F]
</IfModule>
# END iThemes Security – Do not modify or remove this line# BEGIN WordPress
<IfModule mod_rewrite.c>
RewriteEngine on
RewriteCond %{HTTP_HOST} ^(www.)?thesturdytree.com$
RewriteCond %{REQUEST_URI} !^/wp/
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)$ /wp/$1
RewriteRule ^(/)?$ wp/index.php [L]
</IfModule>
# END WordPress# RWS edits and additions
ErrorDocument 404 /404.html
<files wp-config.php>
order allow,deny
deny from all
</files># Block the include-only files
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteRule ^wp-admin/includes/ – [F,L]
RewriteRule !^wp-includes/ – [S=3]
RewriteRule ^wp-includes/[^/]+\.php$ – [F,L]
RewriteRule ^wp-includes/js/tinymce/langs/.+\.php – [F,L]
RewriteRule ^wp-includes/theme-compat/ – [F,L]
</IfModule>
# prevent directory browsing
Options All -Indexes
# protect .htaccess file
<Files .htaccess>
order allow,deny
deny from all
</Files>Geez!
What happens if you comment out
ErrorDocument 404 /404.html
That’s keeping the WP 404 page from loading.That line is totally ineffective. I added it today as a test but it made no difference. Same without.
Edit:
I just noticed the WP directory has an almost duplicate .htaccess. I see no 404.html anywhere except /.-
This reply was modified 7 years, 1 month ago by
sturdy2.
I was talking only about the .htaccess file in the root of the WP directory. Is there another one?
I have 2 similar .htaccess files. One under domain root and the other under /wp. Latter copy below:
# BEGIN iThemes Security - Do not modify or remove this line # iThemes Security Config Details: 2 # Enable HackRepair.com's blacklist feature - Security > Settings > Banned Users > Default Blacklist # Start HackRepair.com Blacklist RewriteEngine on # Start Abuse Agent Blocking RewriteCond %{HTTP_USER_AGENT} "^Mozilla.*Indy" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^Mozilla.*NEWT" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^$" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^Maxthon$" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^SeaMonkey$" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^Acunetix" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^binlar" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^BlackWidow" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^Bolt 0" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^BOT for JCE" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^Bot mailto\:craftbot@yahoo\.com" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^casper" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^checkprivacy" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^ChinaClaw" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^clshttp" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^cmsworldmap" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^Custo" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^Default Browser 0" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^diavol" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^DIIbot" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^DISCo" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^dotbot" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^Download Demon" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^eCatch" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^EirGrabber" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^EmailCollector" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^EmailSiphon" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^EmailWolf" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^Express WebPictures" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^extract" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^ExtractorPro" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^EyeNetIE" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^feedfinder" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^FHscan" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^FlashGet" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^flicky" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^g00g1e" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^GetRight" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^GetWeb\!" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^Go\!Zilla" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^Go\-Ahead\-Got\-It" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^grab" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^GrabNet" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^Grafula" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^harvest" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^HMView" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^Image Stripper" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^Image Sucker" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^InterGET" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^Internet Ninja" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^InternetSeer\.com" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^jakarta" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^Java" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^JetCar" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^JOC Web Spider" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^kanagawa" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^kmccrew" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^larbin" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^LeechFTP" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^libwww" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^Mass Downloader" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^microsoft\.url" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^MIDown tool" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^miner" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^Mister PiX" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^MSFrontPage" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^Navroad" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^NearSite" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^Net Vampire" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^NetAnts" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^NetSpider" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^NetZIP" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^nutch" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^Octopus" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^Offline Explorer" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^Offline Navigator" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^PageGrabber" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^Papa Foto" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^pavuk" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^pcBrowser" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^PeoplePal" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^planetwork" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^psbot" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^purebot" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^pycurl" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^RealDownload" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^ReGet" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^Rippers 0" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^sitecheck\.internetseer\.com" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^SiteSnagger" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^skygrid" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^SmartDownload" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^sucker" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^SuperBot" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^SuperHTTP" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^Surfbot" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^tAkeOut" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^Teleport Pro" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^Toata dragostea mea pentru diavola" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^turnit" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^vikspider" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^VoidEYE" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^Web Image Collector" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^WebAuto" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^WebBandit" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^WebCopier" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^WebFetch" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^WebGo IS" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^WebLeacher" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^WebReaper" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^WebSauger" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^Website eXtractor" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^Website Quester" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^WebStripper" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^WebWhacker" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^WebZIP" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^Widow" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^WPScan" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^WWW\-Mechanize" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^WWWOFFLE" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^Xaldon WebSpider" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^Zeus" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "^zmeu" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "360Spider" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "AhrefsBot" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "CazoodleBot" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "discobot" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "EasouSpider" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "ecxi" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "GT\:\:WWW" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "heritrix" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "HTTP\:\:Lite" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "HTTrack" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "ia_archiver" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "id\-search" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "IDBot" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "Indy Library" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "IRLbot" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "ISC Systems iRc Search 2\.1" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "LinksCrawler" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "LinksManager\.com_bot" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "linkwalker" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "lwp\-trivial" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "MFC_Tear_Sample" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "Microsoft URL Control" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "Missigua Locator" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "MJ12bot" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "panscient\.com" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "PECL\:\:HTTP" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "PHPCrawl" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "PleaseCrawl" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "SBIder" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "SearchmetricsBot" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "SeznamBot" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "Snoopy" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "Steeler" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "URI\:\:Fetch" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "urllib" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "Web Sucker" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "webalta" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "WebCollage" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "Wells Search II" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "WEP Search" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "XoviBot" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "YisouSpider" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "zermelo" [NC,OR] RewriteCond %{HTTP_USER_AGENT} "ZyBorg" [NC,OR] # End Abuse Agent Blocking # Start Abuse HTTP Referrer Blocking RewriteCond %{HTTP_REFERER} "^https?://(?:[^/]+\.)?semalt\.com" [NC,OR] RewriteCond %{HTTP_REFERER} "^https?://(?:[^/]+\.)?kambasoft\.com" [NC,OR] RewriteCond %{HTTP_REFERER} "^https?://(?:[^/]+\.)?savetubevideo\.com" [NC] # End Abuse HTTP Referrer Blocking RewriteRule ^.* - [F,L] # End HackRepair.com Blacklist, https://pastebin.com/u/hackrepair <IfModule mod_rewrite.c> RewriteEngine On # Reduce Comment Spam - Security > Settings > WordPress Tweaks > Comment Spam RewriteCond %{REQUEST_METHOD} POST RewriteCond %{REQUEST_URI} /wp-comments-post\.php$ RewriteCond %{HTTP_USER_AGENT} ^$ [OR] RewriteCond %{HTTP_REFERER} !^https?://(([^/]+\.)?thesturdytree\.com|jetpack\.wordpress\.com/jetpack-comment)(/|$) [NC] RewriteRule ^.* - [F] </IfModule> # Protect System Files - Security > Settings > System Tweaks > System Files <files .htaccess> <IfModule mod_authz_core.c> Require all denied </IfModule> <IfModule !mod_authz_core.c> Order allow,deny Deny from all </IfModule> </files> <files readme.html> <IfModule mod_authz_core.c> Require all denied </IfModule> <IfModule !mod_authz_core.c> Order allow,deny Deny from all </IfModule> </files> <files readme.txt> <IfModule mod_authz_core.c> Require all denied </IfModule> <IfModule !mod_authz_core.c> Order allow,deny Deny from all </IfModule> </files> <files wp-config.php> <IfModule mod_authz_core.c> Require all denied </IfModule> <IfModule !mod_authz_core.c> Order allow,deny Deny from all </IfModule> </files> # Disable Directory Browsing - Security > Settings > System Tweaks > Directory Browsing Options -Indexes <IfModule mod_rewrite.c> RewriteEngine On # Protect System Files - Security > Settings > System Tweaks > System Files RewriteRule ^wp-admin/install\.php$ - [F] RewriteRule ^wp-admin/includes/ - [F] RewriteRule !^wp-includes/ - [S=3] RewriteRule ^wp-includes/[^/]+\.php$ - [F] RewriteRule ^wp-includes/js/tinymce/langs/.+\.php - [F] RewriteRule ^wp-includes/theme-compat/ - [F] # Disable PHP in Uploads - Security > Settings > System Tweaks > PHP in Uploads RewriteRule ^wp\-content/uploads/.*\.(?:php[1-7]?|pht|phtml?|phps)$ - [NC,F] # Disable PHP in Plugins - Security > Settings > System Tweaks > PHP in Plugins RewriteRule ^wp\-content/plugins/.*\.(?:php[1-7]?|pht|phtml?|phps)$ - [NC,F] # Disable PHP in Themes - Security > Settings > System Tweaks > PHP in Themes RewriteRule ^wp\-content/themes/.*\.(?:php[1-7]?|pht|phtml?|phps)$ - [NC,F] # Filter Request Methods - Security > Settings > System Tweaks > Request Methods RewriteCond %{REQUEST_METHOD} ^(TRACE|DELETE|TRACK) [NC] RewriteRule ^.* - [F] # Filter Suspicious Query Strings in the URL - Security > Settings > System Tweaks > Suspicious Query Strings RewriteCond %{QUERY_STRING} \.\.\/ [OR] RewriteCond %{QUERY_STRING} \.(bash|git|hg|log|svn|swp|cvs) [NC,OR] RewriteCond %{QUERY_STRING} etc/passwd [NC,OR] RewriteCond %{QUERY_STRING} boot\.ini [NC,OR] RewriteCond %{QUERY_STRING} ftp: [NC,OR] RewriteCond %{QUERY_STRING} https?: [NC,OR] RewriteCond %{QUERY_STRING} (<|%3C)script(>|%3E) [NC,OR] RewriteCond %{QUERY_STRING} mosConfig_[a-zA-Z_]{1,21}(=|%3D) [NC,OR] RewriteCond %{QUERY_STRING} base64_decode\( [NC,OR] RewriteCond %{QUERY_STRING} %24&x [NC,OR] RewriteCond %{QUERY_STRING} 127\.0 [NC,OR] RewriteCond %{QUERY_STRING} (globals|encode|localhost|loopback) [NC,OR] RewriteCond %{QUERY_STRING} (request|concat|insert|union|declare) [NC,OR] RewriteCond %{QUERY_STRING} %[01][0-9A-F] [NC] RewriteCond %{QUERY_STRING} !^loggedout=true RewriteCond %{QUERY_STRING} !^action=jetpack-sso RewriteCond %{QUERY_STRING} !^action=rp RewriteCond %{HTTP_COOKIE} !wordpress_logged_in_ RewriteCond %{HTTP_REFERER} !^https://maps\.googleapis\.com RewriteRule ^.* - [F] # Filter Non-English Characters - Security > Settings > System Tweaks > Non-English Characters RewriteCond %{QUERY_STRING} %[A-F][0-9A-F] [NC] RewriteRule ^.* - [F] </IfModule> # END iThemes Security - Do not modify or remove this line # BEGIN WordPress <IfModule mod_rewrite.c> RewriteEngine On RewriteBase /wp/ RewriteRule ^index\.php$ - [L] RewriteCond %{REQUEST_FILENAME} !-f RewriteCond %{REQUEST_FILENAME} !-d RewriteRule . /wp/index.php [L] </IfModule> # END WordPress
I can see that two files may be problematic so I have removed the version from / and the site still seems okay. But I see a change in last URL visited by these guests but still no 404s. Now, the last site visited is always listed as root so it now appears that all are being redirected from /wiki.
Thanks again…
-
This reply was modified 7 years, 1 month ago by
- The topic ‘Unknown URLs not producing 404s’ is closed to new replies.