CloudFlare Bot Fight Mode Conflict

Hi @davidginzberg

I hope you are doing well.

The Link Checker is receiving some nice updates, the current version is using this specific user agent:

$request_args         = array(
			'timeout'    => $timeout,
			'user-agent' => 'Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)', //masquerade as IE 7
			'aa' => 1024 * 5,
		);

I suggest you bypass this UA in Cloudflare, it also uses your server IP to run the tests.

Best Regards
Patrick Freitas

Thanks for your reply @wpmudevsupport12.

Please note the following from my original post:

Please note: This issue is specific to third-party sites blocking Broken Link Checker traffic, so any solution that requires changing CloudFlare settings likely isn’t workable.

Because this issue involves being blocked by sites controlled by a third party, your suggestion is unfortunately not going to work here.

I’d like to request that someone at the WPMU Dev team submit the user agent details you’ve listed above to CloudFlare using The CloudFlare Bot Verification Form so that all users of Broken Link Checker can benefit from whitelisting.

And I’m happy to hear that there are updates coming. If you have a link to a blog post or press release detailing them I’d be happy to take a look to see if any of them will help.

Thank you,
David

Hi @davidginzberg

I hope you are doing well.

I am afraid we use a Generic User-agent, however, we do have some upcoming updates for the Broken Link Checker plugin and I passed this ticket to the plugin team in case we update the UA we can submit the rest to CF.

Best Regards
Patrick Freitas

Thanks for the update @wpmudevsupport12,

I’m running into the same issue on multiple sites now, so if a future release adds an identifiable UA, or a way to set one in the WP Admin settings, that would be very helpful. For now, the best workaround we’ve found is manually checking links that get a 503, but that’s not a sustainable solution long-term. If there’s a better solution you can offer for unblocking these links, I’d welcome the advice.

Thank You,
David

Hi @davidginzberg

I appreciate your feedback and I’m sorry for all the inconvenience the issue is causing.

For now, I can’t offer better solution I’m afraid. As my collague mentioned, there’s some serous update being prepared and the way plugin works (code/service-wise) will be completely revamped to modern standards and with ways opening for future changes/improvement. Unfortunately, I don’t have ETA at this moment.

Kind regards,
Adam

Hello @davidginzberg ,

We haven’t heard from you for a while now, so it looks like you don’t have any more questions for us.

Feel free to re-open this ticket if needed.

Kind regards
Kasia