Is host level page caching a problem or just plugins?

Glad to help.

Basically anything that turns a dynamic page response into a static response, will break plugins like Blackhole. I haven’t tested every page cache available, but do keep a list based on user feedback, it can be found here. Anything that is not on the list basically hasn’t been tested. If you would like to test, there are steps on the plugin’s Installation tab.

I hope this helps, let me know if I can provide any further information.

I completed parts 1 and 2 on your Check if the plugin is working page. Using a VPN, I found the link in the source code and went to the link. I got no feedback in the browser on that first visit or on any other visit, while the VPN was running.

When I went to:
Admin > Blackhole Pro > Bad Bots

I saw the request I’d made via the VPN browser. So, the plugin was logging it, but it wasn’t having an affect on my hitting the page.

Does that sound like a scenario where page caching is having a negative effect?

Yeah basically if the plugin doesn’t work when your page cache is enabled, it pretty much means the cache is stopping dynamic plugins like Blackhole from working properly. Some aspects may work, like the behind scenes logging, but when the page is static, there can be no dynamic blocking/response.