Issue with S3/CDN/Spaces

  • Resolved opicron

    (@opicron)


    5 years ago

    When using S3 CDN or Digital Ocean spaces we face an issue that the local file is not existing.

    Due to this the get_attached_file returns an https:// link to the CDN. But in lrsync_core.php line 863 returns false while the file actually exists (on CDN).

    Could we adjust the filecheck to do an header (not download) file check when the get_attached_file returns an offsite location? Or please provide an hook where we can overrule the default file_exist functionality.

    $header = get_headers($imagepath[0], true);
$filesize = $header['Content-Length'];

    Thank you in advance!

Viewing 3 replies - 1 through 3 (of 3 total)
  • Plugin Author Jordy Meow

    (@tigroumeow)

    Hi @opicron,

    I actually started to replace the

    file_exists

    checks by a custom function that will call a filter (and then let you handle the check), but there is another issue…

    The plugin does a lot of checks like: does the file exist? Yes? Then it checks its size and compares it. If different, it deletes the file and creates a new one. But in this case, that will not work at all.

    Thing is if you are using S3 or remote storage, WordPress doesn’t really work with those files anymore. All the existing plugins which handle this are half-cooked, only handle the uploads well (and hopefully, the deletes), but it’s more complex than this, and all the file operations should be handled. Some plugins work with EXIFs, some other regenerate thumbnails, rename or move files around, and in the case of WP/LR Sync… keep then synchronized.

    Ideally, WordPress will add its own functions for dealing with files one day. Then, those plugins would hook on those. Right now, they just hack through WordPress and they leave it full of holes, and for me, that sounds like… quite a mess.

    It’s better to keep those files safe next to the WordPress install, and CDN solutions are extremely good at making all this fast.

    I can push the new plugin with the change, and you can try to play with it, but I doubt the updates of files will go smoothly.

    You could try to modify WP/LR Sync to make it works in your case, by, as I did, replacing the system calls by functions in WP/LR Sync (which are using apply_filters), and try to make a little plugin for WP/LR Sync + the remote upload plugin you are using. I am pretty sure you could do it but you would need quite a lot of patience ??

    I will definitely include the changes in the plugin if it’s clean, and you could upload that plugin on the repository too. That could be interesting too.

    Thread Starter opicron

    (@opicron)

    Well, what I want to do in the custom hook is download the file. The everything will work fine again :).

    On the server an script is running which will re-upload the file to S3 after optimizing.

    And an day later the wordpress upload folder is clean of one day old images.

    I am sure many plugins will break with S3, that is an given. But this case I can handle with your custom hook and re-download before doing anything.

    Thank you for the extensive response. Always much appreciated! My extension to your fantastic wplr works really well. It is not really ready for release, but I can share the code with you if you wish.

    Plugin Author Jordy Meow

    (@tigroumeow)

    Hi @opicron,

    That’s awesome ?? Don’t hesitate to contact me directly. I have already added a few hooks in my code (the ones you should use) and I would like to make sure it works with what you have been doing. Then I can release that version, and maybe you can release that extension you made, that could be fun. You are definitely not the only one using S3 with WP/LR Sync, but I have no idea what the others do (maybe they don’t “resync”, and just use WP/LR Sync to upload).

Viewing 3 replies - 1 through 3 (of 3 total)
  • The topic ‘Issue with S3/CDN/Spaces’ is closed to new replies.