David E. Smith

So, I just upgraded to 3.2.2 and the problem has returned. Clearing my environment’s Varnish caches, restarting php-fpm, clearing browser caches, none of these seem to resolve my issue. I’m not entirely sure the issue ever was fixed, but I’m still trying to narrow things down. The only thing of which I’m reasonably certain, is that if I revert to 3.1.6, I have no issues.

Closing out my part of this issue: It looks like, in my case at least, the problem appears to have been a server-side issue. (My PHP-FPM opcode cache seemed to have parts of different versions of the plugin stuck in memory; restarting the php-fpm process cleared up my problems.)

Bug report sent. I toggled as many themes and plugins as I could without breaking the site completely, none of that seemed to help.

Since downgrading NextGEN Gallery to 3.1.6, making no other changes, resolved the problem on my production site, I really think it’s an issue with this plugin specifically.

If you think you’ll need access to the site’s back-end, reach out to me via the email in the ticket I sent, as it can take a couple days to grant access to our org’s single-sign-on system.

Downgrading my sites to 3.1.6 seems to have solved the issue, so it’s likely that whatever’s happening was introduced just after that.

I’m seeing something very similar on my sites. They’re only on 3.1.11. (Tried upgrading one to 3.1.17, and downgrading to 3.1.7, and neither helped.) Several of the JS and CSS links are having ‘/wp-content/plugins/’ replaced with, in my case, ‘/var/www/html/wordpress-shared/plugins-shared/’ (a directory where most of our sites’ plugins live, so we don’t have to manage hundreds of copies of them). This was just noticed today. We updated our production sites to 3.1.11 about a week ago.

https://my.domain/var/www/html/wordpress-shared/plugins-shared/nextgen-gallery/products/photocrati_nextgen/modules/nextgen_admin/static/bootstrap/css/bootstrap-tooltip.css?ver=3.1.11

Key environment details: Red Hat Enterprise Linux 7, PHP 7.2.16 or so, MariaDB 10.2.something. Can provide any additional needed info on request.

I’m still using it on ~400 locally-hosted sites. This is one of those instances where it’s not broke, so they don’t fix it. ??

Since it’s a drop-in, not a standard plugin, it’s always amusing to me to see “less than 10” installs. The real number is probably many thousands, but since drop-ins don’t get reported back to .org through their telemetry we may never know.

Done!

https://github.com/michaelryanmcneill/shibboleth/issues/54

As I look things over, I think I may be conflating two separate issues, which we happened to discover at about the same time.

The first issue is the one I originally reported, the one with the ‘regex’ column (on at least one site). That one, you’ve already said a bugfix is forthcoming, and thank you for that.

The second one is how some sites didn’t evidently get their schemas upgraded properly. Since you already have code to handle upgrades, I wonder if it’d be worth doing a general checkup on things as part of every plugin update.

(Another horrifying possibility that just occurred to me: When does the schema update normally trigger? Some of the sites where I’m seeing the old schema are test sites, where it’s entirely possible nobody has accessed them in a few months…)

I’ve spot-checked several of the 85 sites we have using Redirection. In general, it looks like sites that we’ve had for a few years still have the ‘old’ schema, and younger sites have the new/correct schema, but I can’t immediately find a 100% verifiable pattern. If you updated the schema in 3.0, my wild guess would be that something (a previous schema change, maybe?) specific to older sites caused the schema update to fail.

Based on the datestamps in this plugin’s change log, and our usual upgrade cycles, we probably went from 2.10.1, straight to 3.2. But that would have been for all of our sites, so that doesn’t immediately explain why the upgrade worked on most sites but not all. And no, I don’t have logs from February to explain the failure ??

Wonder if it’d be worth adding a function that triggers on every plugin upgrade, that re-verifies (and corrects if necessary) the database layout?

First, the “old” PHP 5.3/MariaDB 10.1 environment. I looked both at a mysqldump, and ran a SHOW CREATE TABLE query, and they appear to be identical.

CREATE TABLE med_redirection_items (
id int(11) unsigned NOT NULL AUTO_INCREMENT,
url mediumtext NOT NULL,
regex int(11) unsigned NOT NULL DEFAULT ‘0’,
position int(11) unsigned NOT NULL DEFAULT ‘0’,
last_count int(10) unsigned NOT NULL DEFAULT ‘0’,
last_access datetime NOT NULL,
group_id int(11) NOT NULL DEFAULT ‘0’,
status enum(‘enabled’,’disabled’) NOT NULL DEFAULT ‘enabled’,
action_type varchar(20) NOT NULL,
action_code int(11) unsigned NOT NULL,
action_data mediumtext,
match_type varchar(20) NOT NULL,
title text,
PRIMARY KEY (id),
KEY url (url(200)),
KEY status (status),
KEY regex (regex),
KEY group_idpos (group_id,position),
KEY group (group_id)
) ENGINE=InnoDB AUTO_INCREMENT=564 DEFAULT CHARSET=utf8

Then, I migrate the data to our new environment. The migration is a shell script, that does a mysqldump from the old server, then an import to the new one. After that process is done, here’s the output of a SHOW CREATE TABLE in the new PHP 7.2/MariaDB 10.2 environment:

CREATE TABLE med_redirection_items (
id int(11) unsigned NOT NULL AUTO_INCREMENT,
url mediumtext NOT NULL,
regex int(11) unsigned NOT NULL DEFAULT 0,
position int(11) unsigned NOT NULL DEFAULT 0,
last_count int(10) unsigned NOT NULL DEFAULT 0,
last_access datetime NOT NULL,
group_id int(11) NOT NULL DEFAULT 0,
status enum(‘enabled’,’disabled’) NOT NULL DEFAULT ‘enabled’,
action_type varchar(20) NOT NULL,
action_code int(11) unsigned NOT NULL,
action_data mediumtext DEFAULT NULL,
match_type varchar(20) NOT NULL,
title varchar(50) DEFAULT NULL,
PRIMARY KEY (id),
KEY url (url(200)),
KEY status (status),
KEY regex (regex),
KEY group_idpos (group_id,position),
KEY group (group_id)
) ENGINE=InnoDB AUTO_INCREMENT=551 DEFAULT CHARSET=utf8

So, in the old one, the default values for the int columns are shown as ‘0’ but for the new version they’re just 0 (maybe some sort of typing, or maybe just a change in how the output is displayed). The action_data column seems to have picked up a DEFAULT NULL that I’m not sure where it came from, and the title column has changed types altogether. I’m so very confused right now, it looks like somehow the database itself is sabotaging me.

I’m trying to find documentation on any sort of implicit type changes between MariaDB 10.1 and 10.2. Haven’t found it yet, but the day is young.

(The auto-increment counters being different isn’t surprising since I did the last instance of copying this site from the current production environment, to the new one, a few days ago.)

Updating this plugin to use explicit values for every field when doing an INSERT will probably remedy my woes, and help future-proof the plugin from any other silliness some future MySQL-alike might have lurking.

On a different site, recently moved between our old and new hosting environments, we got a variation on a theme:

Error: Unable to add new redirect (29): Field ‘last_access’ doesn’t have a default value
Raw: {“code”:”redirect”,”message”:”Unable to add new redirect”,”data”:{“status”:400,”error_code”:29,”wpdb”:”Field ‘last_access’ doesn’t have a default value”}}

In this case, the “old” site was PHP 5.3 and MariaDB 10.1; the “new” one is the same as above (PHP 7.2, MariaDB 10.2). The site was moved with a simple mysqldump. Maybe there’s a quirk in 10.2, or different handling of default values?

(Internally, things are even more complicated with HyperDB, but we’re only using it for read replication balancing, not for anything like sharding. That shouldn’t matter here, but …)

Unfortunately, I ended up creating my own plugin for this (it’s actually part of a much larger in-house plugin). I do wish there were a way to actively modify an uploaded theme or plugin, but I understand I’ve got a really niche use-case.

(Yes, it basically did boil down to instantiating my own copy of Plugin_Upgrader or Theme_Upgrader, feeding it the appropriate arguments, and doing $upgrader->run().)

Almost anything with ‘automatic’ in the description will be a hard sell to our hyper-conservative upper management (for the case of a mistaken commit followed by a late-night automatic update that WSOD’s the site). But I’ll dig into it a bit more, and if nothing else I may steal borrow some code.

Now that I know where to look, it looks like it might be as simple as:

Get the location of the uploaded file (probably somewhere in /tmp)
Inspect the file, correct those things which can be corrected
Create an instance of WP_Upgrader and feed the file to $upgrader->run()

We’re trying to replace an older plugin called “WP Easy Uploader” that we install on dev/test sites, so that developers can upload code. That plugin doesn’t honor, or intentionally ignores, the fact that DISALLOW_FILE_MODS is set in our sites’ wp-config.php files, which is useful in our case. But it doesn’t do any real validation, it will let you do unwise things like upload over parts of core, and it doesn’t work on PHP 7.

Since we want to replace (part of) that functionality anyway, might as well try to improve upon things. Specifically, when you grab a zip file from Bitbucket or Github, the file often has silliness that we want to correct. (Example: Bitbucket usually names the top-level directory orgname-slug-commit, but WordPress expects just the slug, for instance. If you don’t fix this you end up with a half-dozen nearly-identical copies of the same plugin.)

So, my goal is eventually to write an in-house plugin/theme uploader and updater, that detects and resolves (when possible) the odd things that are likely to crop up in uploaded files, then hands them off to the built-in update routines. If I can do the cleanup on an already-uploaded file, that would save a manual step, but if that’s not possible I could settle for detecting such things, and stopping the update with a suitable end-user notification.

Handling uploads isn’t too hard, and cleaning up an uploaded zip file isn’t too hard; I just need to work out how to take the (potentially-modified) zip and hand it off to the internal plugin and theme updaters.