• Oliver Campion

    (@domainsupport)


    Is anyone else seeing errors like this in v6.1 (I think since v6.0.2 and v6.0.3) … ?

    [09-Nov-2022 11:28:28 UTC] Cron unschedule event error for hook: do_pings, Error code: could_not_set, Error message: The cron event list could not be saved., Data: {“schedule”:false,”args”:[]}
    [09-Nov-2022 18:05:03 UTC] Cron unschedule event error for hook: wordfence_processAttackData, Error code: could_not_set, Error message: The cron event list could not be saved., Data: {“schedule”:false,”args”:[]}

    Thank you,

    Oliver

Viewing 15 replies - 31 through 45 (of 50 total)
  • I had this problem again on my site and noticed that the cron in alloptions stored in the object cache was different than the cron in wp_options in the database.
    These values are set by functions such as update_option, but since exclusive control is not performed, if multiple processes/threads (not limited to concurrent cron) set the values at the same time, different values may set.
    update_option has a check that does not update unless the value changes, but this check is based on the object cache. Therefore, the same value as in the database bypasses the check, resulting in 0 updates where previously reported.

    So I added the following code to the theme’s functions.php and the error disappeared.

    wp_cache_delete( 'alloptions', 'options' );

    Performance will be affected, but I will see how it goes in this state for now.

    Thread Starter Oliver Campion

    (@domainsupport)

    Hi @wakabayashi0827

    We’ve managed to eradicate the issue from our servers. But not by changing anything in WordPress core. I don’t actually think that this is a bug in core. I think that it’s always happened but in v6.1 an update was introduced that detected the problem and logged a PHP notice accordingly.

    There may be many nuanced causes but I believe they all can be generalised by being a database issue. Be that the database is too busy to save the CRON schedule, the database has been disconnected or something similar.

    Other users may disagree with this analysis but I stand by it.

    So, to answer your questions …

    1. No, but I believe that it raises a requirement for WordPress to better detect and warn when there is an ongoing or intermittent issue with the database (when the database is overloaded / underpowered / disconnected).
    2. I have conducted many, many tests across several sites and my analysis is that the cause is a overloaded / underpowered / disconnected database.
    3. So … in our main case the cause was that our site was receiving a lot of DDOS attacks. WordPress CRON out of the box is triggered not by the system but instead by the visitors requests (when a visitor requests a page, if there is a CRON schedule due to run, it is run). So for us, this was often happening during a DDOS attack when the database was being overloaded. We moved our CRON to be fired by the system (hourly) and disabled the default method for firing CRON. This has (so far) resolved the issue entirely for us although we fully expect to see a notice in the future when an hourly firing of CRON coincides again with a DDOS attack.
    4. The issue itself does not affect sites in my opinion. CRON events are still run successfully; all that is not successful is the saving of the CRON schedule which means that recurring events are unable to be re-scheduled. This is not a problem so long as the missing CRON schedules are implemented correctly as they will be added again when the plugin, WordPress core or whatever added them in the first place notices that they are missing from the schedule. However, it may well be that the database issue preventing the schedule from being saved also affects whatever the CRON event(s) are doing.
    5. The issue has a ticket on trac but I don’t believe a “fix” will be forthcoming simply because it’s a symptom of a larger (database) issue rather than a problem in itself.

    I hope this helps to explain where I’m at with this ??

    Oliver

    Thread Starter Oliver Campion

    (@domainsupport)

    @krstarica Check your code is looking to make sure the CRON events are scheduled with a if (!wp_next_scheduled ( ... )) { wp_schedule_event( ... ) } and if not add the event. If this happens then your events will be added again when your code notices they are missing. https://developer.www.ads-software.com/reference/functions/wp_schedule_event/#comment-2277

    Thread Starter Oliver Campion

    (@domainsupport)

    Yes, we moved ours to the plugin class __construct() function. We decided any minor loss of efficiency was worth it to make sure events were not missed. Also, I think (although correct me if I’m wrong) that this is how WordPress does it with core events.

    Hello,

    Same error messages on different events for very low traffic sites. I see no reason to have DB overload issues. Everything seems to be running fine, and the missed events keep running later on. Per site, I have these errors once or twice per day, and I have 36 events running per site.

    I just migrated from 5.5.x to 6.2.2 on a shared server. I see no fix for this in this thread, except this workaround

    wp_cache_delete( 'alloptions', 'options' );

    Any good with the above? Any risk to have it enabled for a couple of days?

    [24-May-2023 23:12:53 UTC] Cron reschedule event error for hook: action_scheduler_run_queue, Error code: could_not_set, Error message: The cron event list could not be saved., Data: {"schedule":"every_minute","args":["WP Cron"],"interval":60}
    [24-May-2023 23:12:53 UTC] Cron reschedule event error for hook: wp_privacy_delete_old_export_files, Error code: could_not_set, Error message: The cron event list could not be saved., Data: {"schedule":"hourly","args":[],"interval":3600}
    [25-May-2023 07:01:18 UTC] Cron reschedule event error for hook: action_scheduler_run_queue, Error code: could_not_set, Error message: The cron event list could not be saved., Data: {"schedule":"every_minute","args":["WP Cron"],"interval":60}

    Thread Starter Oliver Campion

    (@domainsupport)

    @swissspaceboy Have you double checked to be sure? What’s in your /var/log/mysql/error.log or equivalent at the time of the PHP notices? Our notices were occurring during times where the server was getting spiked from hacking scans and were unrelated to the regular traffic.

    Thread Starter Oliver Campion

    (@domainsupport)

    OK, well to re-iterate everything that can theoretically cause the could_not_set error notice …

    1. The $option parameter is empty
      This will never happen (it’s always set, to cron when the CRON is being saved)
    2. The old option value is the same as the new option value
      This suggests the cron job is running more than once (the chances of this happening are almost non-existent as they would require the transient to be the same which would require the two concurrent requests to have been received within the same microsecond)
    3. The database insert query?$wpdb->update()?fails

    So I realise it’s a bitter pill … but everything about this issue points to the database for whatever reason.

    It also explains why a single solution cannot be found because there are several reasons that can cause an issue with the database. Perhaps the database has become disconnected (see this comment on trac). Perhaps the database is overwhelmed / out of RAM / has too many connections. Perhaps something has locked the database up. Who knows.

    If you’re not seeing anything in your database log … maybe enable more verbose logging and see exactly what was going on with the database at that time. For example, if the database had disconnected then there wouldn’t be an error in the database log.

    And just because your site has low legitimate traffic, doesn’t mean that the database can’t be overwhelmed. All WordPress sites are regularly scanned for vulnerabilities by hackers and each scan is an enormous strain on a server with hundreds or (more often) thousands of requests that can look (and act) like a DDoS attack. We see it all the time and it’s why we developed Deny All Firewall to only allow content that exists to be requested.

    I have zillions of cron error messages in my error log which makes it difficult to debug other stuff. I have duplicate core wp cron jobs for 4 different hooks. Each one has an event stuck in 2018. I cannot delete them because they are protected and if I pause one, it pauses all the duplicates as well. So I thought, “I will outsmart it and change the frequency on one to annually so that at least it will quit filling up my log.”… well that just created an additional duplicate. Sigh.

    My short fix has been to comment out lines 129-139 and 156-165 of wpcron.php. That at least cleans up my log file. But I don’t want all these duplicates running or even listed. Any ideas for getting rid of them? I am not experienced enough to know how they are getting in there in the first place. The ones I am having trouble with are:

    • wp_version_check
    • wp_update_themes
    • wp_update_plugins
    • wp_privacy_delete_old_export_files
    Thread Starter Oliver Campion

    (@domainsupport)

    @integritivity If you are getting duplicate cron events then that is a different issue and you should start your own thread.

    This thread is only regarding the could_not_set cron PHP error notice.

    Oliver

    ok, time for some concrete actions after 10 pages of this thread.

    How do we debug this? Can WP add some extra traces in the code when this error is logged? Or maybe just do a retry of the same action? 99% that the second call would pass.

    We need to do something to get rid of this. Either there is a bug and we fix it, or we obfuscate the error if there is no real impact. I don’t want to have errors that cannot be resolved.

    So how do we proceed? I am ready to be a beta tester…

    Didier.

    Thread Starter Oliver Campion

    (@domainsupport)

    @swissspaceboy Unless anyone can provide evidence that what I’ve stated in this comment is not true then I’m afraid it’s unlikely that there will be a “fix” or that it will be removed.

    A fix suggests there’s something wrong with WordPress core and I believe the issue is caused when the database, not WordPress, has an issue. Obfuscating the notice would also not be a helpful suggestion since, in my opinion, the issue is appearing when the site has a problem … with the database.

    We have still yet to see another occurrence of this notice since we moved WP CRON to be fired by the system and blocked public access to /wp-cron.php.

    @domainsupport I think this is not only database issue.

    1. Duplicate CRON causes difference between cache and database, and Cron unschedule event message is logged.
      (described in https://www.ads-software.com/support/topic/cron-unschedule-event-error-for-hook/page/9/)
    2. There is a mechanism to prevent duplicate CRON, but it doesn’t seem to work perfectly.
    3. As you explained, changing CRON to be fired by the system will almost certainly prevent duplicate crons.
      (I have confirmed that the same results are obtained on my site.)
    4. But for some sites (like using hosting services) CRON cannot be changed to fire from the system.
    Thread Starter Oliver Campion

    (@domainsupport)

    @me240 @j3gaming I looked into duplicate CRON causing the issue right at the beginning but ruled it out because the transient would require the two requests to be fired at exactly the same millisecond. An unlikely scenario that even if it were possible would not be so frequently observed.

    However … transients are stored in the database … you see where I’m going here? ??

    If the doing_cron transient cannot be stored by one CRON request because the database is unavailable (overwhelmed / out of RAM / has too many connections, locked, disconnected, etc, etc) then it cannot be checked by another CRON request and is another symptom of the same underlying issue … namely … the database.

    I see all valid replies from everyone, but no concrete solution.

    Can WordPress not handle this use case where a database might have some issues at one moment in time? As an end user, what can I do on this error? I see no errors in my database. So it is fine that WP logs an error, but what do we do with it? Currently, it is only polluting our logs (at least in my case).

    If the problem is database related, we cannot count on a fix on that side: too many databases, too many hosters, shared servers, etc. WP would need to handle this use case from the application level.

    Any ideas on catching this error, and do something else so that the cron event runs again or it is scheduled correctly for the next time?

    We cannot keep this “critical” error for years…

    Didier.

    Thread Starter Oliver Campion

    (@domainsupport)

    @swissspaceboy I understand your frustration.

    The problem is that it’s very likely that each database issue is different.

    Take a look at this comment which attempts to explain one of them (when the database connection is lost). If I’m right (and I suspect that I am) then this entire thread is just discussing the symptom of a completely different, multi-facetted problem; namely any database issue.

    As a user, I would contact your ISP and ask them …

    • How you can gain access to your verbose MySQL error logs?
    • How many concurrent connections are allowed to your database?
    • Can they see if you exceeding this?
    • How much RAM is given to your database?
    • Can they see if you exceeding that?
    • Has your MySQL service had any interruptions?
    • Has your MySQL connection disconnected in PHP?

    I’m not 100% sure if all of these questions are valid (and it’s not an exhaustive list) but if you can get timestamps for anything highlighted from them and marry any database issues up to the PHP error notices … you may have something you can try to resolve.

Viewing 15 replies - 31 through 45 (of 50 total)
  • The topic ‘Cron unschedule event error for hook’ is closed to new replies.