Custom Filters

What’s the actual URL you put in?

The example is $myurl = 'https://example.com/newest/'; but obviously that’s not YOUR actual URL ??

The actual url is https://www.mychesco.com/?call_custom_simple_rss=1&csrp_cat=-17434&csrp_thumbnail_size=735×400

BTW, I also tried setting up a cron job using:

curl -X PURGE https://example.com

I set it to run every hour on the hour, and it does seem to do anything.

Are you still there? Is there a solution?

Please try to be more patient. If it helps to put this in perspective for you, I spent 6 hours at a hospital yesterday so no, I didn’t take a look at anything about this.

The actual url is https://www.xxx.com/?call_custom_simple_rss=1&csrp_cat=-17434&csrp_thumbnail_size=735×400

A couple things here.

Those are ‘ugly’ URLs and a number of Varnish caches won’t even cache that.

Second, can you confirm your WHOLE code looked like this:

function my_varnish_urls( $urls, $postid ) {
	$myurl = 'https://www.xxx.com/?call_custom_simple_rss=1&csrp_cat=-17434&csrp_thumbnail_size=735×400';
	array_push( $urls, $myurl ) ;
return $urls;
}
add_filter( 'vhp_purge_urls', 'my_varnish_urls' );

I removed the actual domain in case you felt like keeping it private. But replace xxx with the domain name.

• This reply was modified 2 years, 7 months ago by Ipstenu (Mika Epstein).

I’m sorry to hear of your hospitalization.

For me, It doesn’t matter if it is private or not. I remove the domain out of habit.

Yes, that it is the code that was placed in functions.php. I simply cut and paste the code I found at https://github.com/Ipstenu/varnish-http-purge/wiki/Custom-Filters (which I discovered in the plugin’s FAQs) and replaced with the url.

The header shows it is cacheable. I’m on DreamHost if that helps.

strict-transport-security: max-age=31536000
vary: Accept-Encoding
via: 1.1 varnish (Varnish/6.5)
x-cache: HIT
x-cacheable: YES
x-powered-by: DreamPress
x-varnish: 31134428 30871014

Huh. DreamHost should be set up that an & URL like yours isn’t cached. THAT SAID…

I ran a curl on your site and it says you have CloudFlare. curl -I DOMAIN will show you and on that…

Are you using CloudFlare?

I am using cloudflare but that page is setup to bypass cloudflare’s cache… which is confirmed in the header.

Okay. So I think the problem is … you can’t actually use the domain like that. Cloudflare intercepts the command because it’s sent to the domain, and happens after the code checks to add the ‘default’ flushing.

That said, I haven’t had a reason or time to fully test out how it works when CloudFlare’s in the mix. Adding a cache proxy layer on top of a cache proxy layer always comes with headaches ??

And weirdly when I check your domain with Curl I still see all the expected cloudflare headers. I DO see cf-chl-bypass is set to 1, but that’s the captcha bypass IIRC.

My best thought right now is change the https://example.com/ to https://YOURIPADDRESS/

If you log in to your site and go to the Varnish settings pages, you’ll see where it lists an IP at the bottom. That’s the IP the plugin uses. Sadly it uses it only when it’s adding on the default things to purge, and doesn’t touch the filtered URLs since some people want to pass params to an external site… If that does work, I’ll see about running a filter check for the Varnish IP if the domain added is the same as the domain the site is on, but that may be a bit since it’s not a direct logic flow :/ Lots of cases to keep an eye out for.

…you can’t actually use the domain like that? …if the domain added is the same as the domain the site is on? I’m not really sure that I understand what you mean.

Regardless, this seems to have exposed another issue… search results (example: https://www.mychesco.com/?s=coatesville).

Response Headers:

age: 45
alt-svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400
cache-control: must-revalidate, public, max-age=300, stale-while-revalidate=360, stale-if-error=43200
cf-apo-via: origin,qs
cf-cache-status: BYPASS
cf-edge-cache: cache,platform=wordpress
cf-ray: 6f8ceae24ffe1a0f-EWR
content-encoding: br
content-type: text/html; charset=UTF-8
date: Fri, 08 Apr 2022 18:03:03 GMT
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
expires: Fri, 08 Apr 2022 19:02:15 GMT
link: <https://www.mychesco.com/wp-json/>; rel="https://api.w.org/"
nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=tWfLiJnmFR0TMjd%2F4TKlk8dAAnoboHpIVxSs5%2FHbUN6NOtLpqys9xSa8zyM9W3S6Z3WbsrA9e39B7G4PGPpdcGWGqSSt8GCG9K5JPBt8OnYolK94xJRjHl0ozXsM5PQ%2FFeOmqtAUINWvtSKsiSw%3D"}],"group":"cf-nel","max_age":604800}
server: cloudflare
strict-transport-security: max-age=31536000
vary: Accept-Encoding
via: 1.1 varnish (Varnish/6.5)
x-cache: HIT
x-cacheable: YES:Forced
x-powered-by: DreamPress
x-varnish: 31268004 27626825

So, it looks like Varnish is caching search results which will stick around essentially until I do a full site purge. That’s not right, is it?

I’m not at all sure how Cloudflare is part of the issue here. If I do a full site purge of Vanish, without touching Cloudflare, the Ugly URLs (both created by the site search and Custom Simple Rss plugin) do clear and update… again showing that Cloudflare is being bypassed.

So, it looks like Varnish is caching search results which will stick around essentially until I do a full site purge. That’s not right, is it?

Or until it ‘times out’ which IIRC is a few hours. That’s by-design, since search pages are very ‘heavy’ resource wise, so caching them will speed up your site.

I’m not at all sure how Cloudflare is part of the issue here. If I do a full site purge of Vanish, without touching Cloudflare, the Ugly URLs (both created by the site search and Custom Simple Rss plugin) do clear and update… again showing that Cloudflare is being bypassed.

You’re doing the purge through the plugin, right?

If so, it’s because that function in the plugin was specifically coded to work around things like Cloudflare. However that code does NOT extend to any additional URLs you’ve put in ??

The reason that Cloudflare ‘gets in the way’ is that you pointed your DNS to them. So to get to your server it goes:

domain -> Cloudflare -> DreamHost

The plugin internally compensates as best it can for those kinds of intermediary proxies.

FWIW this is why your CURL didn’t work.

curl -X PURGE https://example.com can’t work BECUASE CloudFlare gets in thw way.

Which is mentioned here: https://github.com/Ipstenu/varnish-http-purge/wiki/varnish#purging

You would want:

curl -X PURGE "https://123.45.67.89/" -H "host:www.example.com" where that IP (123.45.67/89) is replaced with YOUR IP.

Same problem as the filter’s having ?? CF is ‘in between’ the command and the domain.

• This reply was modified 2 years, 7 months ago by Ipstenu (Mika Epstein).

I had already tried the curl -X PURGE "https://123.45.67.89/" -H "host:www.example.com"… of course, with my IP address. It did not work either.

I temporarily turned Cloudflare off, then purged Varnish…

Response Headers for https://www.mychesco.com/?call_custom_simple_rss=1&csrp_cat=-17434&csrp_thumbnail_size=735%C3%97400

accept-ranges: bytes
age: 5
cache-control: must-revalidate, public, max-age=300, stale-while-revalidate=360, stale-if-error=43200
cf-edge-cache: cache,platform=wordpress
content-encoding: gzip
content-length: 9368
content-type: text/xml; charset=utf-8
date: Fri, 08 Apr 2022 19:37:57 GMT
expires: Sun, 08 May 2022 19:37:49 GMT
server: Apache
strict-transport-security: max-age=31536000
vary: Accept-Encoding
via: 1.1 varnish (Varnish/6.5)
x-cache: HIT
x-cacheable: YES
x-powered-by: DreamPress
x-varnish: 26476987 28574484

Still cacheable.

If the curl -X didn’t work, then it’s not the plugin. Seriously, that’s really all the plugin is doing.

Also it being cachable isn’t the issue, it’s the cache not clearing.

But… looking at your example of the headers, notice “age: 5”? That means the page is 5 seconds old.

So that actually looks like it DID purge, and you’re on a new version.