All visitors being occasionally redirected to 127.0.0.1 on home page

Hi @hausinteractive

Can you please cross-check the WP security > Dashboard > Permanent blocklist.

Is there any IP blocked? If yes do there is any cache plugin on which may cache the homepage being redirected to 127.0.0.1

WP security > Firewall > Internet bots if below option is enabled please disable it. and cross check if it solves the issue

Ban POST requests that have a blank user-agent and referer:

Regards

Thanks @hjogiupdraftplus. Answers:

Can you please cross-check the WP security > Dashboard > Permanent blocklist. Is there any IP blocked? If yes do there is any cache plugin on which may cache the homepage being redirected to 127.0.0.1

There are no IP addresses blocked now, or when I first reported the issue.

WP security > Firewall > Internet bots if below option is enabled please disable it. and cross check if it solves the issue

This option was not previously enabled and continues to be disabled.

Ban POST requests that have a blank user-agent and referer:

This part of your reply seems to have gotten truncated but “Ban POST requests” was not previously enabled and continues to be disabled.

There is no strict pattern to the problem; it does not happen at the same time of day, nor with the same increment between days, nor for the same amount of time while it is happening. We do see it in the wild roughly 2-6 times per month.

Our plan for the next time this is triggered will be to clear the site cache manually and see if this response is at least being cached. If so, the time after that we can disable the security plugin and then clear the cache to isolate this to AIOS.

More as we have it, but we are definitely open to more feedback in the interim.

Hi @hausinteractive

Do you have the W3 Total cache plugin is installed?

127.0.0.1 for all site visitors front pages redirected can only due to cache + IP permanently blocked by AIOS either due to Spam comments or due to Ban POST requests that have a blank user-agent and referer

OR you have 404 detection have locked out the IP and after cache flush first time that blocked IP visiting the site.

If you can check the database aiowps_login_lockdown do have that IPs blocked at the same time it will have release time also as it do not permanently block IP but temp lockout IP access. WP security > Dashboard > Lockout IP list will show only during temp lockout those IPs not after release.

Regards

Thanks @hjogiupdraftplus,

Do you have the W3 Total cache plugin is installed?

We do. Due to the traffic to the site we can not disable this. We are planning to manually clear the cache next time we see this issue. If the locked down home page is cached it should become accessible right away.

127.0.0.1 for all site visitors front pages redirected can only due to cache + IP permanently blocked by AIOS either due to Spam comments

Spam Prevention -> Comment Spam -> “Detect spambots posting comments” is enabled AND

Spam Prevention -> “Comment spam IP monitoring” is enabled with 1,805 blocked IPs BUT

The website home page does not have a comment form so it should not be able to trigger an IP block AND

These settings do make any mention of redirecting bad IPs to 127.0.0.1

or due to Ban POST requests that have a blank user-agent and referer

Firewall -> Internet Bots -> “Ban POST Requests that have a blank user-agent and referrer” is NOT enabled and has never been enabled.

OR you have 404 detection have locked out the IP and after cache flush first time that blocked IP visiting the site.

Brute Force -> 404 Detection is NOT enabled and was never enabled. We have changed the 404 lockout redirect URL to 127.0.0.2 and left the 404 detection to OFF. This will help test to see if the feature is “active” even though it is not “enabled” in the admin. If it is still active even though it’s set to OFF we’ll see this new redirect URL in the logs.

If you can check the database?aiowps_login_lockdown?do have that IPs blocked at the same time it will have release time also as it do not permanently block IP but temp lockout IP access. WP security > Dashboard > Lockout IP list will show only during temp lockout those IPs not after release.

We have checked this DB table and there is one IP block from earlier this year that does not coincide with any of our 127.0.0.1 redirect timestamps. (this lockout is not even on the same day as a 127.0.0.1 redirect outage)

Thanks!

Hi @hausinteractive

Spam Prevention -> “Comment spam IP monitoring” is enabled with 1,805 blocked IPs

Those should be in {db_prefix}_aiowps_permanent_block table and in AIOS should be showing in WP security > Dashboard > Permanent block list

W3Total cache do have Memcache enabled?

Is there any proxy server installed where your own server IP might get blocked?

Here according to me cache is cleared then > visit from blocked IP of home page might create cache to the redirect 127.0.0.1. and applied for all.

But how is this done have to check in more details. As this should not be the general case.

Regards

Thanks @hjogiupdraftplus

Spam Prevention -> “Comment spam IP monitoring” is enabled with 1,805 blocked IPs. Those should be in?{db_prefix}_aiowps_permanent_block?table and in AIOS should be showing in WP security > Dashboard > Permanent block list

Yes. We see 6 addresses in this table that attempted to access the site on the most recent date this problem occurred. None of the timestamps coincide with the 127.0.0.1 redirect, however.

W3Total cache do have Memcache enabled?

Yes.

Is there any proxy server installed where your own server IP might get blocked?

No.

Here according to me cache is cleared then > visit from blocked IP of home page might create cache to the redirect 127.0.0.1. and applied for all. But how is this done have to check in more details. As this should not be the general case.

This 127.0.0.1 redirect happened again yesterday. We were able to detect it quickly and cleared the site cache. As expected, the home page was restored immediately. This is not surprising, but we have confirmed that when the problem is triggered it is being cached.

Unfortunately our monitor only checks the site every N minutes, so alerts are usually delayed by a few minutes, meaning we don’t get exact timestamps of when the redirect is triggered.

You mentioned certain POST requests potentially being problematic. This IP address raises an eyebrow due to:

? It being a POST
? It happening around the time of the redirect trigger
? It generating odd response codes (302 for the first request, 411 for the second request)

168.227.229.96 www.mediaplaynews.com - [18/Nov/2024:04:19:14 -0800] "POST /wp-comments-post.php HTTP/1.1" 302 - "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"

7 seconds later:

168.227.229.96 www.mediaplaynews.com - [18/Nov/2024:04:19:21 -0800] "POST /wp-comments-post.php HTTP/1.1" 411 239 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"

This IP address is NOT in the {db_prefix}_aiowps_permanent_block table.

Hi @hausinteractive,

Ok, can you disable for W3 Total cache memcache, So it will be Disk enhanced.

As memcache is not making issue want to make sure.

I will try contact W3 Total cache plugin also so we know the why an such issue.

302 code might be it is being redirected to 127.0.0.1

411 code server refused the request.

Regards

Thanks @hjogiupdraftplus, we will switch to ‘disk enhanced’ and monitor progress. We would prefer not to run in this mode long term as memcache provides significant performance that we need.

Follow-up question: What about memcache makes you suspect it is the culprit?

Hi @hausinteractive,

I have in local the memcached server not installed and tested for w3Total cache disk enhanced. Purge cache > Then access from blocked ip so redirect to 127.0.0.1 and then try access from non blocked IP. It loads the page and do not redirect,

So just want to make sure It is not memcache issue. I can cross check in local by installing memcache if you do not want to disable.

Regards

Thanks @hjogiupdraftplus. If you’re willing to install memcached I’d love to see what your testing reveals. I think we’ll be able to find that answer much quicker than the W3TC folks would.

I think we’ll survive for a while with disk caching, and it sounds like we might need to anyway if the problem ends up being with W3TC and their interaction with memcached.

Hi @hausinteractive,

Ok, let me test here with memcached enabled.

Regards