Search Spam – Reclaiming Crawl Budget
-
Hi, like many others we have recently experienced our fair share of search spam targeting the core search functionality of WordPress (?s= & /search/). 300k+ requests in the last week or so.
The Yoast Crawl optimization options and most specifically the Internal site search cleanup options go some way to helping the issue. However, if possible we would like to go further.
Our primary goal at the moment relates to Yoast’s automatic redirection of filtered/blocked searchs to 301 redirect to our homepage. We wondered if it might be possible to specifiy the redirect page in these instances? We ask because one option may be to redirect to “blackhole” page and then potentially block their IP based on visiting that “blackhole” page; something easily achieved using Wordfence.
This would vastly reduce the number of spam requests hitting the server. Right now, allowing these requests to respond with any page, be it the search results page, our homepage or 404 page:
1) Uses valuable server resources (300k+ requests)
2) May potentially skew Google Analytics (300k + visits to our homepage)
3) Continue to use Google Crawl budget. Despite not being indexed, our understanding is they still have considerable impact on Google Crawl Budget. We’d like to try prevent/reduce this impact if possible.
N.B. as a side note it would be nice to have a some transparency around the “common spam patterns” setting in Yoast to better understand what that filter might be doing?
- The topic ‘Search Spam – Reclaiming Crawl Budget’ is closed to new replies.
