Upload New Media Intermittent Failure

  • Resolved greatmatter

    (@greatmatter)


    4 years, 3 months ago

    We have suddenly started to experience an intermittent (inconsistent) error when uploading media:
    “Post-processing of the image failed likely because the server is busy or does not have enough resources. Uploading a smaller image may help. Suggested maximum size is 2500 pixels.”

    and when it’s a non-image (PDF, ZIP):
    “Unexpected response from the server. The file may have been uploaded successfully. Check in the Media Library or reload the page.”

    The corresponding error in the apache log is:
    [Fri Jul 31 04:42:04.308166 2020] [proxy_fcgi:error] [pid XXXX:tid XXXXXXXXXXXXXXX] (70008)Partial results are valid but processing is incomplete: [client XXX.XXX.XXX.XXX:0] AH01075: Error dispatching request to : (reading input brigade), referer: https://XXXXXXX.XXX/wp-admin/media-new.php

    • When the upload begins, the progress bar reaches 100%. Approximately 30-40 seconds later, the error appears.
    • This error occurs approximately 30% of the time.
    • The error can be replicated across different browsers and logins.
    • File size or file type do not matter; the error appears in tiny and large files alike.
    • If a file fails to upload, and we try again with the same file, the file will upload at the same rate of success (approximately 30%)

    The following steps have been taken to attempt to fix this:

    • Increase PHP memory_limit
    • Disabling reqtimeout module (Apache)
    • Increasing max execution time in PHP
    • Updating PHP/Apache (currently 7.4.8 FPM and 2.4.43, respectively)
    • Increasing server resources (and all other server settings accordingly)
    • Restarting Apache/PHP
    • Restarting the server
    • Swearing loudly (very cathartic)
    • Telling the client to be patient (not so cathartic)
    • Disable/reinstall/reenable ImageMagick (thought it could be a silent fail with thumbnail creation, given the fact it fails with both image and PDFs)
    • Disabling plugins
    • Googling until tabs thought they were one with the tab bar

    Additional potentially useful info

    • The uploads directory is NOT divided by date. Don’t know why it was setup this way, but switching it to date-based directories is not feasible due to some hacky scripts the last guy dumped me with
    • There are 271,309 files in the uploads directory – HOWEVER we attempted trying a completely empty uploads folder, and it still produced the same error.
    • This is a brand new problem. It wasn’t happening 2 days ago, and this install is about 5 years old.
Viewing 1 replies (of 1 total)
  • Thread Starter greatmatter

    (@greatmatter)

    We’ve discovered the source of the issue; I’d like to outline it here for posterity and just in case someone finds themselves in an edge case.

    We have Google Cloud Armor implemented at the edge in front of our Load Balancer.

    Removing/disabling Cloud Armor immediately resolved this issue.

    We’re going to open a ticket with Google–but just in case anyone experiences this…try messing with Cloud Armor, or even your Web Application Firewall settings.

Viewing 1 replies (of 1 total)
  • The topic ‘Upload New Media Intermittent Failure’ is closed to new replies.