dicknet

Thanks, Radgeek!

@tom, I noticed that in your earlier message, so had already done that. Didn’t seem to help in this case.

I followed all the suggestions above, but all my tweets simply say “Failed”. Status in the log is Failed as well. No explanation, just Failed. Not helpful.

I’m sorry, but I’m having trouble understanding your answer.

Given what you said above, I don’t understand why the option “Tweet automatically when content is generated by other plugins…” even exists. Does the code have a list of plugins it will play nice with and ones that it won’t?

That issue aside, the fact remains that the Wordtwit window on each post in question says that Wordtwit tweeted it, when in fact it didn’t. I would call this a bug.

With new version it now connects with Twitter OK, but will not tweet automatically. Will only tweet manually. I will start a new thread for this.

Thank you so much. This plugin is so much more reliable than anything else I’ve tried; I feel lost without it.

Sounds like a good place to start.

Pending a resolution, I have disabled the plugin and reverted to using Twitterfeed.

Thank you – so it’s not just me.

Makes me think Twitter is messing with the interface again.

Any chance of getting a fix for this, BraveNewCode?

No, unfortunately not. As a workaround, I put in a cron job to clean it out by brute force every night. Not a great solution, but it was the best I could come up with.

Wow, kellogg9, thanks for all the out-of-the-box ideas. Forces me to look at things differently, which is good.

I was thinking the Sitemap Auto-Caching was something completely different (i.e., IT would make a sitemap). I can’t see a way to immediately cache a new post (and it would have to be immediate because these bots are fast). HOWEVER, most human users are going to come to the main page first, so auto-caching that should relieve some load as well as giving users faster response than they otherwise may have gotten.

The CDN is, frankly, beyond my capabilities. I can barely keep my plain vanilla VPS together.

The throttle idea got me thinking, though. I found a plugin called Wordfence that does a whole lot of useful stuff. For one thing it can automatically block Googlebot impersonators. It can also throttle or block users that hit the site too hard – and this is highly configurable. So I’ve installed that and am evaluating it. If it works out I’ll install it everywhere.

So thanks for the suggestions and for making me look at this differently.

Bots, how to explain the bots? With most of my sites, new posts get posted on Twitter by one mechanism or another. Apparently there are a large number of bots (scrapers, scammers, and some legit) that are poised waiting for new tweets with URLs in them. When they find them they all make a beeline for the URLs in question. It wouldn’t be a problem, but for the fact they all hit at once, bringing my server to its knees. I have banned many of the most egregious offenders and those that seem to offer me little or no benefit. The spikes are lower and less frequent, but they’re still there.

But this brings me back to my original thought: if the first bot causes the page to be cached and subsequent bots are served a copy, it SHOULDN’T bring the server to its knees. So what’s really going on?

One thing I was wondering is if there is a setting to cache a page as soon as it’s posted. So if traffic comes along shortly for that page it will already be cached. I’m not seeing it in the options, but some of the terminology is confusing and I could be missing it.

I’m having the same problem. Didn’t used to be a problem. Checked the feed; no dups there.

There’s a plugin called “FeedWordPress Duplicate Post Filter”; it did not work for me.

Any suggestions would be welcome. And I’m not married to FeedWordPress; if anyone knows of a similar plugin that works better, I’d be willing to try it.

Yes, the ‘?p=999’ flavor. But you’re right, it’s at the top. The URL at the bottom is the permalink.

Toying with the idea of a cron job to do a brute force clearing of the cache every night. Would prefer a proper fix, but the cron job would at least keep the storage under control.

FWIW, here is the cron record from the wp_options table in the database. As you can see there is an entry for garbage collection. Whether or not it’s formatted correctly, I have no idea:

a:5:{i:1351963068;a:1:{s:45:"ws_plugin__qcache_garbage_collector__schedule";a:1:{s:32:"40cd750bba9870f18aada2478b24840a";a:3:{s:8:"schedule";s:6:"hourly";s:4:"args";a:0:{}s:8:"interval";i:3600;}}}i:1351990192;a:2:{s:17:"wp_update_plugins";a:1:{s:32:"40cd750bba9870f18aada2478b24840a";a:3:{s:8:"schedule";s:10:"twicedaily";s:4:"args";a:0:{}s:8:"interval";i:43200;}}s:16:"wp_update_themes";a:1:{s:32:"40cd750bba9870f18aada2478b24840a";a:3:{s:8:"schedule";s:10:"twicedaily";s:4:"args";a:0:{}s:8:"interval";i:43200;}}}i:1351990239;a:1:{s:16:"wp_version_check";a:1:{s:32:"40cd750bba9870f18aada2478b24840a";a:3:{s:8:"schedule";s:10:"twicedaily";s:4:"args";a:0:{}s:8:"interval";i:43200;}}}i:1352045366;a:1:{s:19:"wp_scheduled_delete";a:1:{s:32:"40cd750bba9870f18aada2478b24840a";a:3:{s:8:"schedule";s:5:"daily";s:4:"args";a:0:{}s:8:"interval";i:86400;}}}s:7:"version";i:2;}

To answer the last question first, here is the result of the loopback test: Loopback Response: SUCCESS!! Loopback Requests Work!

So apparently it’s not that.

This issue came up in the context of moving sites from an old hosting provider to a new one. The time to do that is a function of size, and I noticed the sites using caching were disproportionately large. Right now only about half the sites are moved. But I have checked sites on both the old and the new server; both are exhibiting this problem. So we may be able to rule out some oddball setting on the server.

I’m not seeing a pattern in what’s in the old cache pages. I saw pages and posts, and something that looked like search results. One thing that seemed odd is that the URL I saw in the cache file is the WP ‘default’ form, not the permalink. I suspect that’s intentional, though.

One thing I did do today was to disable wp-cron and, instead, run it from a cron job every 10 minutes. Too soon to say if that made any difference.

HTH and makes sense.