Robot exclusion list
-
As far as I am aware the robot exclusion list is populated on installation and won’t get updated because you can also manually edit it. Is that initial list sourced from somewhere public and if so can you point there? I fear the list filters fewer and fewer bots as time goes on and would like to update it manually at a low frequency.
Viewing 3 replies - 1 through 3 (of 3 total)
Viewing 3 replies - 1 through 3 (of 3 total)
- The topic ‘Robot exclusion list’ is closed to new replies.