A few issues with the plugin

Hello @coldrealms65

Ability to use complicated path names. I like to name folders for the site due to subdomains, but /var/cache/npp/subdomain.sitename/ folder is invalid and throw an incorrect error about root folders.

The Nginx Cache Directory option has restrictions on the paths you can use to prevent accidental deletions or harm to critical system files. By default, certain paths, like ‘/home’ and other vital system directories, are blocked to safeguard your system’s stability and prevent data loss.

While this might limit your options, it ensures your system’s security. Recommended directories to choose from, such as ‘/dev/shm/’ or ‘/var/cache/’, which are commonly used for caching purposes and are generally safer.

I will check the folder structer format you mentioned that throw in error even in safe path /var/cache/, added to-do list for v2.0.5, thank you for your feedback.

• The Key format is very restrictive. I like to have separate caches for mobile and desktop so my key setup is like this:
  fastcgi_cache_key “$scheme$request_method$host$mobile_device_type$request_uri$is_args$args”

Honestly, I’ve already tried to handle this case, but due to the limitations in determining and adjusting the plugin code’s regex to match the user’s actual fastcgi_cache_key format, it’s not an easy task.

To gather accurate data for the ADVANCED tab, NPP currently use common fixed format $scheme$request_method$host$request_uri via regex;

if (preg_match('/KEY:\s+httpsGET(.+)/', $content, $matches)) {

This regex is designed to match URLs that follow a specific structure, typically aligned with the format $scheme$request_method$host$request_uri. However, given the nature of your fastcgi_cache_key setup, which includes variables like $mobile_device_type, it becomes challenging to create a universal regex that accommodates all possible combinations of custom cache keys. But I will re-check for any easy solution.

I assume the shell_exec requirement is based on needing that other script setup in your setup for users with separate php and nginx users. For those of us who don’t need that workaround we should be able to re-disable shell_exec. Unless there is another reason you need it?

Mainly, shell_exec is essential for core functionality—Cache Preloading—because the Cache Preload action relies on the native wget command. Additionally, shell_exec is necessary for important pre-checks that NPP performs exclusively on Nginx web servers running on Linux-based systems; otherwise, it will disable itself.

• This reply was modified 4 months ago by Hasan CALISIR. Reason: add more details

Thanks for the response, looking forward to potential changes.

With regards to the following:

Honestly, I’ve already tried to handle this case, but due to the limitations in determining and adjusting the plugin code’s regex to match the user’s actual fastcgi_cache_key format, it’s not an easy task.

To gather accurate data for the ADVANCED tab, NPP currently use common fixed format?$scheme$request_method$host$request_uri?via regex;

Could you add an advanced option to have the user supply their own regex? A toggle or even fallback that would use your base key format and still allow the user to set their own for their specific use?

Explain that any non-standard regex will not receive official support.

Mainly,?shell_exec?is essential for core functionality—Cache Preloading—because the Cache Preload action relies on the native?wget?command. Additionally,?shell_exec?is necessary for important pre-checks that NPP performs exclusively on Nginx web servers running on Linux-based systems; otherwise, it will disable itself.

As shell_exec is such a serious security issue there should be a way to get around this. For the time being I am a fan of options so perhaps a toggle to disable those features that need shell_exec? Have in the setup to enable shell_exec while configuring the plugin so the tests can run. Inform them that the preload requires shell_exec so the user can make that choice.

In the meantime I will be looking for a way to do the preload without shell_exec.

New question – does the plugin tie into the action hooks for wp rocket or any other cache systems so that when they trigger a cache purge so does your plugin? If not, I know that Adame and his team from wp rocket are really good about helping plugin devs hook into their actions

Hello @coldrealms65 ,

Could you add an advanced option to have the user supply their own regex? A toggle or even fallback that would use your base key format and still allow the user to set their own for their specific use?

Explain that any non-standard regex will not receive official support.

I checked you fastcgi_cache_key and If I understand correctly, you use a map directive along with $http_user_agent to define a custom variable $mobile_device_type, which is then included in your fastcgi_cache_key to separate different versions of cached content for mobile and desktop devices.

map $http_user_agent $mobile_device_type {
    default "";
    "~*Android" "mobile";  # Match Android devices
    "~*iPhone" "mobile";   # Match iPhone devices
    "~*iPad" "mobile";     # Match iPad devices (optional)
    "~*Windows Phone" "mobile";  # Match Windows Phone
    "~*Windows NT" "desktop";     # Match Windows desktop
    "~*Macintosh" "desktop";       # Match Mac desktop
    "~*Linux" "desktop";           # Match Linux desktop
}

$scheme$request_method$host$mobile_device_type$request_uri$is_args$args

When you visit the ADVANCED tab, NPP parses the cached content, generates data, draws a table on the fly with generated data, and allows for single Purge and Preload actions. In this process, parsing the correct URL from the fastcgi_cache_key is crucial for Preload action. While parsing the cached content URL’s, NPP must rely on a fixed regex.

• Getting user custom fastcgi_cache_key as a plugin option and auto generating regex and manipulating plugin code according to custom key format on the fly is not efficient.
• Trying to fixed catch-all regex to cover all possible custom key formats also not efficient.

For example the actual URL of cached content:

www.example.com/path/to/resource

With $scheme$request_method$host$request_uri the line we need to parse is:

KEY: httpsGETwww.example.com/path/to/resource

With $scheme$request_method$host$mobile_device_type$request_uri$is_args$args

~ It becomes:

KEY: httpsGETwww.example.commobile/path/to/resource/?utm_source=mobile&referrer=homepage

Even though I assumed you named it “mobile” while mapping, it could be anything, and this is just one simple cache_key format variation among many.

Could you add an advanced option to have the user supply their own regex?

If I understand correctly asking as a plugin option, however, this approach would require manipulating the plugin code dynamically on the fly, which not be efficient.

As shell_exec is such a serious security issue there should be a way to get around this. For the time being I am a fan of options so perhaps a toggle to disable those features that need shell_exec? Have in the setup to enable shell_exec while configuring the plugin so the tests can run. Inform them that the preload requires shell_exec so the user can make that choice.

You can check how and why shell_exec is used in source code here https://github.com/psaux-it/nginx-fastcgi-cache-purge-and-preload/tree/main

In the meantime I will be looking for a way to do the preload without shell_exec.

Thank you for your contributions! You can create a pull request on GitHub, and you are always welcome to contribute.

New question – does the plugin tie into the action hooks for wp rocket or any other cache systems so that when they trigger a cache purge so does your plugin? If not, I know that Adame and his team from wp rocket are really good about helping plugin devs hook into their actions

I noted this and try to implement in v2.0.5

• This reply was modified 4 months ago by Hasan CALISIR.
• This reply was modified 4 months ago by Hasan CALISIR.
• This reply was modified 4 months ago by Hasan CALISIR. Reason: typo

As a non-plugin developer I have a few, probably silly, questions:

If the plugin generates the file list dynamically what is the reason the regex isn’t stored, then pulled, from a database option with a hardcoded fallback like other saved options? Why is it only hardcoded?

For the shell_exec: curl seems to work, but again as a plugin/ coder novice I really have no idea the why and why not, but wouldn’t this do it?

$output = preload_cache_with_curl(
    $fdomain, 
    $tmp_path, 
    $nginx_cache_limit_rate, 
    $nginx_cache_wait, 
    $nginx_cache_reject_regex, 
    $nginx_cache_reject_extension
);
function preload_cache_with_curl($url, $tmp_path, $limit_rate, $wait_time, $reject_regex, $reject_extension) {
    $ch = curl_init();
    
    curl_setopt($ch, CURLOPT_URL, $url);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
    curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 5);
    curl_setopt($ch, CURLOPT_TIMEOUT, 5);
    curl_setopt($ch, CURLOPT_USERAGENT, NPPP_USER_AGENT);
    
    // Custom headers and options
    $headers = [
        'Cache-Control: no-cache',
        'Pragma: no-cache'
    ];
    curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
    
    // Execute and get the response
    $response = curl_exec($ch);
    
    if (curl_errno($ch)) {
        $error_msg = curl_error($ch);
        curl_close($ch);
        return $error_msg;
    }
    
    curl_close($ch);
    return $response;
}

If I am completely off base, as a newb I accept that I am just trying to be helpful :), please feel free to disregard, it won’t hurt my feelings.

Hello?@coldrealms65

Having that technical mindset truly makes a difference, regardless of development experience, and all of your approaches are very valuable.

For the shell_exec: curl seems to work, but again as a plugin/ coder novice I really have no idea the why and why not, but wouldn’t this do it?

About using curl instead of wget to get rid of shell_exec: while both can request URLs, wget has a specific advantage for cache preloading in this case. The -r (recursive) option in wget can automatically follow and preload every link on a website, triggering Nginx cache creation for each page it encounters. With curl to preload entire cache, we’d need to specify each URL manually or build additional logic to fetch and follow links, which adds complexity.

Additionally, we use wget with shell_exec withnohup, allowing the process to completely detach from PHP and run in the background simply. With curl, without shell_exec, we’d need to detach the PHP process to run in the background, which also adds complexity.

If the plugin generates the file list dynamically what is the reason the regex isn’t stored, then pulled, from a database option with a hardcoded fallback like other saved options? Why is it only hardcoded?

I understand your approach better now, you offer to ability to Add an Option for Regex in the Database that built by user ~something like that.

// Retrieve user-defined regex from the database, with a hardcoded fallback
$custom_regex = get_option('nppp_custom_cache_regex');
$regex = $custom_regex ? $custom_regex : '/KEY:\s+httpsGET(.+)/';

if (preg_match($regex, $content, $matches)) {

This is easy to implement; however, requiring plugin users to create a custom regex based on their fastcgi_cache_key can be a drawback for non-technical users. Overall, though, It does provide flexibility, so I’ll add this option to v2.0.5.

Hello @coldrealms65

I released version 2.0.5 and implemented the approaches and fixes we discussed. I kindly request your feedback on the new release.

I will be releasing version v2.0.6 soon to address some urgent minor changes, so your feedback before the release would be greatly appreciated.

Best. ~Hasan

Sorry I missed this, have been ill all week.

I’ll have a go at the 2.0.7 version in a bit and get back to you

I hope you’re feeling better now!

I’ve released version v2.0.8 and implemented the approaches and fixes we discussed. I’d greatly appreciate your feedback on this new release before I move forward with version v2.0.9, which will address some urgent minor changes.

Thanks for your time, and I look forward to your thoughts on the update!

Best,~Hasan