Problems with media_sideload_image

  • leadhos

    (@leadhos)


    5 months ago

    Hi,

    We are importing external images to our WordPress page, using media_sideload_image. Mostly working just fine. Problem is that some of the sources have some kind of WAF, basically not responding.

    Inside media_sideload_image, there is a call to function download_url, with a default timeout of 300 seconds.

    Let’s say 50 of our 3000 images fail to download due to the timeout. This results in our import running for 50 timed out images * 300 seconds, equaling 15000 seconds. That is over 4 hours.

    What we want is to limit the timeout of download_url inside media_sideload_image. Is there any way to do this, without straight up copying media_sideload_image, which in turn will result in missing out on future updates?

    Native code for reference:

    function media_sideload_image( $file, $post_id = 0, $desc = null, $return_type = 'html' ) {
    if ( ! empty( $file ) ) {

    $allowed_extensions = array( 'jpg', 'jpeg', 'jpe', 'png', 'gif', 'webp' );

    /**
     * Filters the list of allowed file extensions when sideloading an image from a URL.
     *
     * The default allowed extensions are:
     *
     * - jpg
     * - jpeg
     * - jpe
     * - png
     * - gif
     * - webp
     *
     * @since 5.6.0
     * @since 5.8.0 Added 'webp' to the default list of allowed file extensions.
     *
     * @param string[] $allowed_extensions Array of allowed file extensions.
     * @param string $file The URL of the image to download.
     */
         $allowed_extensions = apply_filters( 'image_sideload_extensions', $allowed_extensions, $file );
    $allowed_extensions = array_map( 'preg_quote', $allowed_extensions );

    // Set variables for storage, fix file filename for query strings.
    preg_match( '/[^\?]+\.(' . implode( '|', $allowed_extensions ) . ')\b/i', $file, $matches );

    if ( ! $matches ) {
    return new WP_Error( 'image_sideload_failed', __( 'Invalid image URL.' ) );
    }

    $file_array = array();
    $file_array['name'] = wp_basename( $matches[0] );

    // Download file to temp location.
    $file_array['tmp_name'] = download_url( $file ); // EXCESSIVE TIMEOUT OF 300 SECONDS!!

    // If error storing temporarily, return the error.
    if ( is_wp_error( $file_array['tmp_name'] ) ) {
    return $file_array['tmp_name'];
    }

    // Do the validation and storage stuff.
    $id = media_handle_sideload( $file_array, $post_id, $desc );

    // If error storing permanently, unlink.
    if ( is_wp_error( $id ) ) {
    @unlink( $file_array['tmp_name'] );
    return $id;
    }

    // Store the original attachment source in meta.
    add_post_meta( $id, '_source_url', $file );

    // If attachment ID was requested, return it.
    if ( 'id' === $return_type ) {
    return $id;
    }

    $src = wp_get_attachment_url( $id );
    }

    // Finally, check to make sure the file has been saved, then return the HTML.
    if ( ! empty( $src ) ) {
    if ( 'src' === $return_type ) {
    return $src;
    }

    $alt = isset( $desc ) ? esc_attr( $desc ) : '';
    $html = "<img src='$src' alt='$alt' />";

    return $html;
    } else {
    return new WP_Error( 'image_sideload_failed' );
    }
    }
    • This topic was modified 5 months ago by leadhos.
    • This topic was modified 5 months ago by leadhos.
Viewing 2 replies - 1 through 2 (of 2 total)
  • Moderator bcworkz

    (@bcworkz)

    You can use the ‘http_request_args’ filter to alter the timeout arg. Be aware that this filter impacts all instances of WP_Http::request(). Your callback is passed the args and the URL to be requested. Between the two you should be able to discern sideload requests from any others where altering the timeout might be inappropriate.

    Mary Hubbard

    (@4thhubbard)

    To limit the timeout of the download_url function without copying the entire media_sideload_image function, you can use a filter to modify the timeout value. The download_url function allows you to set a timeout through its arguments.

    Here’s how you can approach it:

    1. Use the http_request_timeout Filter: You can set a shorter timeout globally for HTTP requests using the http_request_timeout filter. This will affect all HTTP requests, including those made by download_url.
    2. Create a Custom Timeout: Instead of changing the global timeout, you could modify the download_url function call within a custom wrapper function that applies a specific timeout.

    Here’s a way to implement this: Using http_request_timeout Filter

    You can add this code to your theme’s functions.php or a custom plugin:

    add_filter('http_request_timeout', function($timeout) {
    return 10; // Set your desired timeout in seconds
});

    Custom Wrapper for download_url

    Alternatively, if you want to limit the timeout just for your use case without affecting other downloads, you could create a wrapper for download_url:

    function custom_download_url($url, $timeout = 10) {
    $args = array(
        'timeout' => $timeout,
    );
    return download_url($url, $args);
}

// Then modify your media_sideload_image function or create your own
function custom_media_sideload_image($file, $post_id = 0, $desc = null, $return_type = 'html') {
    // ... other code remains the same ...

    // Use the custom download function
    $file_array['tmp_name'] = custom_download_url($file, 10); // Set your custom timeout

    // ... rest of the code remains the same ...
}

    Important Notes

    • If you’re using the filter method, be cautious as it will affect all HTTP requests on your site.
    • If you choose to create a wrapper function, it’s good practice to document it and ensure that you’re aware of any changes in future WordPress updates related to download_url.
    • Always test changes on a staging site before deploying to production to avoid disrupting your live environment.

    This approach allows you to manage the timeout effectively without losing the benefit of updates to the original media_sideload_image function.

Viewing 2 replies - 1 through 2 (of 2 total)
  • You must be logged in to reply to this topic.