hausinteractive

Thanks @hjogiupdraftplus. If you’re willing to install memcached I’d love to see what your testing reveals. I think we’ll be able to find that answer much quicker than the W3TC folks would.

I think we’ll survive for a while with disk caching, and it sounds like we might need to anyway if the problem ends up being with W3TC and their interaction with memcached.

Follow-up question: What about memcache makes you suspect it is the culprit?

Thanks @hjogiupdraftplus, we will switch to ‘disk enhanced’ and monitor progress. We would prefer not to run in this mode long term as memcache provides significant performance that we need.

Thanks @hjogiupdraftplus

Spam Prevention -> “Comment spam IP monitoring” is enabled with 1,805 blocked IPs. Those should be in?{db_prefix}_aiowps_permanent_block?table and in AIOS should be showing in WP security > Dashboard > Permanent block list

Yes. We see 6 addresses in this table that attempted to access the site on the most recent date this problem occurred. None of the timestamps coincide with the 127.0.0.1 redirect, however.

W3Total cache do have Memcache enabled?

Yes.

Is there any proxy server installed where your own server IP might get blocked?

No.

Here according to me cache is cleared then > visit from blocked IP of home page might create cache to the redirect 127.0.0.1. and applied for all. But how is this done have to check in more details. As this should not be the general case.

This 127.0.0.1 redirect happened again yesterday. We were able to detect it quickly and cleared the site cache. As expected, the home page was restored immediately. This is not surprising, but we have confirmed that when the problem is triggered it is being cached.

Unfortunately our monitor only checks the site every N minutes, so alerts are usually delayed by a few minutes, meaning we don’t get exact timestamps of when the redirect is triggered.

You mentioned certain POST requests potentially being problematic. This IP address raises an eyebrow due to:

? It being a POST
? It happening around the time of the redirect trigger
? It generating odd response codes (302 for the first request, 411 for the second request)

168.227.229.96 www.mediaplaynews.com - [18/Nov/2024:04:19:14 -0800] "POST /wp-comments-post.php HTTP/1.1" 302 - "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"

7 seconds later:

168.227.229.96 www.mediaplaynews.com - [18/Nov/2024:04:19:21 -0800] "POST /wp-comments-post.php HTTP/1.1" 411 239 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"

This IP address is NOT in the {db_prefix}_aiowps_permanent_block table.

Thanks @hjogiupdraftplus,

Do you have the W3 Total cache plugin is installed?

We do. Due to the traffic to the site we can not disable this. We are planning to manually clear the cache next time we see this issue. If the locked down home page is cached it should become accessible right away.

127.0.0.1 for all site visitors front pages redirected can only due to cache + IP permanently blocked by AIOS either due to Spam comments

Spam Prevention -> Comment Spam -> “Detect spambots posting comments” is enabled AND

Spam Prevention -> “Comment spam IP monitoring” is enabled with 1,805 blocked IPs BUT

The website home page does not have a comment form so it should not be able to trigger an IP block AND

These settings do make any mention of redirecting bad IPs to 127.0.0.1

or due to Ban POST requests that have a blank user-agent and referer

Firewall -> Internet Bots -> “Ban POST Requests that have a blank user-agent and referrer” is NOT enabled and has never been enabled.

OR you have 404 detection have locked out the IP and after cache flush first time that blocked IP visiting the site.

Brute Force -> 404 Detection is NOT enabled and was never enabled. We have changed the 404 lockout redirect URL to 127.0.0.2 and left the 404 detection to OFF. This will help test to see if the feature is “active” even though it is not “enabled” in the admin. If it is still active even though it’s set to OFF we’ll see this new redirect URL in the logs.

If you can check the database?aiowps_login_lockdown?do have that IPs blocked at the same time it will have release time also as it do not permanently block IP but temp lockout IP access. WP security > Dashboard > Lockout IP list will show only during temp lockout those IPs not after release.

We have checked this DB table and there is one IP block from earlier this year that does not coincide with any of our 127.0.0.1 redirect timestamps. (this lockout is not even on the same day as a 127.0.0.1 redirect outage)

Thanks!

Thanks @hjogiupdraftplus. Answers:

Can you please cross-check the WP security > Dashboard > Permanent blocklist. Is there any IP blocked? If yes do there is any cache plugin on which may cache the homepage being redirected to 127.0.0.1

There are no IP addresses blocked now, or when I first reported the issue.

WP security > Firewall > Internet bots if below option is enabled please disable it. and cross check if it solves the issue

This option was not previously enabled and continues to be disabled.

Ban POST requests that have a blank user-agent and referer:

This part of your reply seems to have gotten truncated but “Ban POST requests” was not previously enabled and continues to be disabled.

There is no strict pattern to the problem; it does not happen at the same time of day, nor with the same increment between days, nor for the same amount of time while it is happening. We do see it in the wild roughly 2-6 times per month.

Our plan for the next time this is triggered will be to clear the site cache manually and see if this response is at least being cached. If so, the time after that we can disable the security plugin and then clear the cache to isolate this to AIOS.

More as we have it, but we are definitely open to more feedback in the interim.

Marking as resolved.

Thanks MBR, I did discover and enable this ‘Rename Login page’ option. Immediately after enabling it the email notifications of blocked IPs stopped arriving.

Since this new login page is not in the site index any attempts to scan and find it will likely trigger the server firewall.

Ultimately this is probably the best solution because it does not fill the DB with miles of logs. If I had continued down this path the next feature request would have been to auto-prune the blocked IP logs so the DB doesn’t get too big. XD

Cheers!

+1 for Curtis’ suggestion. Let’s get this baked into the next version!

Strange, I swore we had memcached and yet there it is: working using the “memcache” option. I now see:

Backend status:
127.0.0.1:11212 => up & running

Cheers.

Hey kpgraham,

I don’t absolutely need, just providing some feedback.

When reviewing you might consider hooking into the rich text editor that WordPress proper uses. That might reduce the weight a bit as well as have these widgets look like the page/post editor which would be nice.

Also: Don’t forget that there are also a lot of Chrome and Safari users out there!

Cheers.

Hi Peter,

I’ve downloaded 0.6.1 and when saving the settings I get this:

ERROR: Memcached cache backend activated but no PHP memcached extension was found.

I see this in our server error logs:

[Fri Mar 08 08:22:26 2013] [error] PHP Fatal error: Call to a member function flush() on a non-object in /directory/wp-content/plugins/wp-ffpc/wp-ffpc-common.php on line 196, referer: https://domain.com/wp-admin/options-general.php?page=wp-ffpcoptions

Additionally, I see this message in the settings:

Backend status: 127.0.0.1:11212 => unknown, please try re-saving settings!

Nothing has changed in our server software from my first post but my test copy of WordPress is now 3.5.1 and our wordpress memcache instance is now on port 11212.

Not sure of Jabawack’s specific issue but this has been my experience with Scissors and Watermark on WordPress 3.5:

Upon Activation these are the messages at the top of the screen:

The plugin Scissors and Watermark need to be set. Please visit the Settings page | Hide Notice.

The plugin generated 242 characters of unexpected output during activation. If you notice “headers already sent” messages, problems with syndication feeds or other issues, try deactivating or removing this plugin.

So the error is of some concern but I forged ahead and clicked on “Settings” and made custom max dimensions for the “Large” size. After the settings were saved this still shows at the top of every WordPress admin page:

The plugin Scissors and Watermark need to be set. Please visit the Settings page | Hide Notice.

Not sure what this means and what isn’t being set. No errors in the apache logs so not much I can do to debug.

Anyway, I uploaded an image larger than my “large” setting image and chose “edit”. In the Scissors and Watermark section I tried Resize, Crop and Rotate:

Resize “worked” but offered none of my presets for thumbnail, medium or large. This is very bad for usability as I don’t want my site admins to have to remember an exact pixel value like 817px to get images sized correctly. It would be nice to allow images to be sized to the max width given, allowing the height to vary or for the height to be cropped.

Crop simply doesn’t work. When I select an area and click “crop” the page refreshes with the original cropping.

Rotate works.

I can confirm ikovacic’s fix. Many thanks!

Quick follow ups:

– Is this going to be baked into a soon-to-be-released update? I would like to vote for this as a working plugin.

– Is this plugin “healthy”? Eg: is this going to continue to get updates?

Hi Peter,

We are running memcached

I am emailing the config to the address listed on your website.

Cheers.

—
Aaron