Dispite “bot ignore option” it still downloads from many bots

Hi, is the above Google IP address “bots” the only IP address you have encountered that the plugin counts even with this feature enabled? Or have you encountered other bots IP addresses in your site?

The other bots I encountered are:

IP/Domain - 207.46.13.41: Bing Bot on this IP address
IP/Domain - 207.46.13.83: Bing Bot on this IP address
IP/Domain - 157.55.39.232: Bing Bot on this IP address
IP/Domain - 5.255.250.5: Web Bot exists on this IP address and Listed in Myip.ms Web Bot list
IP/Domain - 100.43.81.117: Yandex Bot on this IP address
IP/Domain - 5.255.250.65: Web Bot exists on this IP address and Listed in Myip.ms Web Bot list
IP/Domain - 100.43.81.117: Yandex Bot on this IP address
IP/Domain - 100.43.91.105: Yandex Bot on this IP address
IP/Domain - 93.158.161.109: Web Bot exists on this IP address and Listed in Myip.ms Web Bot list
IP/Domain - 100.43.91.105: Yandex Bot on this IP address
IP/Domain - 66.249.79.81: Google Bot on this IP address
IP/Domain - 66.249.79.79: Google Bot on this IP address
IP/Domain - 66.249.79.83: Google Bot on this IP address

I used https://bot.myip.ms/ to check the IP addresses logged.
The Excel file is below:

Log ID	Download ID	Download Title	File URL	Date	IP Address	Country	Name
28	1926	healers	https://healingmentaldisorders.com/wp-content/uploads/2017/08/healingmentaldisorders_com_healers.pdf	11/26/2017 15:54	66.249.79.83	United States	Not Logged In
27	1926	healers	https://healingmentaldisorders.com/wp-content/uploads/2017/08/healingmentaldisorders_com_healers.pdf	11/26/2017 14:16	66.249.79.79	United States	Not Logged In
26	1927	home	https://healingmentaldisorders.com/wp-content/uploads/2017/08/healingmentaldisorders_com_home.pdf	11/26/2017 14:15	66.249.79.81	United States	Not Logged In
25	1930	opinions-1	https://healingmentaldisorders.com/wp-content/uploads/2017/08/healingmentaldisorders_com_opinions.pdf	11/26/2017 10:37	100.43.81.117	United States	Not Logged In
24	1925	gallery	https://healingmentaldisorders.com/wp-content/uploads/2017/09/healingmentaldisorders_com_gallery.pdf	11/26/2017 9:04	84.201.133.82	Russian Federation	Not Logged In
23	1925	gallery	https://healingmentaldisorders.com/wp-content/uploads/2017/09/healingmentaldisorders_com_gallery.pdf	11/26/2017 9:04	100.43.91.105	United States	Not Logged In
22	1929	obsessor	https://healingmentaldisorders.com/wp-content/uploads/2017/08/healingmentaldisorders_com_obsessor.pdf	11/26/2017 8:49	141.8.143.191	United States	Not Logged In
21	1946		https://healingmentaldisorders.com/wp-content/uploads/2017/11/RDP96-00788R001700210016-5.pdf	11/26/2017 8:38	93.158.161.109	Russian Federation	Not Logged In
20	1932	patient	https://healingmentaldisorders.com/wp-content/uploads/2017/08/healingmentaldisorders_com_patient.pdf	11/26/2017 7:46	100.43.81.117	United States	Not Logged In
19	1921	family	https://healingmentaldisorders.com/wp-content/uploads/2017/08/healingmentaldisorders_com_family.pdf	11/26/2017 7:24	100.43.81.117	United States	Not Logged In
18	1934	summary	https://healingmentaldisorders.com/wp-content/uploads/2017/08/healingmentaldisorders_com_summary.pdf	11/26/2017 7:19	100.43.81.117	United States	Not Logged In
17	1933	sleep travels	https://healingmentaldisorders.com/wp-content/uploads/2017/08/healingmentaldisorders_com_sleep.pdf	11/26/2017 3:58	100.43.91.105	United States	Not Logged In
16	1923	FAQ-2	https://healingmentaldisorders.com/wp-content/uploads/2017/09/healingmentaldisorders_com_faq_2.pdf	11/26/2017 1:56	100.43.81.117	United States	Not Logged In
15	1920	healing coil	https://healingmentaldisorders.com/wp-content/uploads/2017/10/healingmentaldisorders_com_coil.pdf	11/26/2017 0:20	100.43.81.117	United States	Not Logged In
14	1917	astral goggles	https://healingmentaldisorders.com/wp-content/uploads/2017/09/healingmentaldisorders_com_astral_goggles.pdf	11/25/2017 23:34	100.43.81.117	United States	Not Logged In
13	1927	home	https://healingmentaldisorders.com/wp-content/uploads/2017/08/healingmentaldisorders_com_home.pdf	11/25/2017 23:05	100.43.81.117	United States	Not Logged In
12	1928	mantras	https://healingmentaldisorders.com/wp-content/uploads/2017/08/healingmentaldisorders_com_mantras.pdf	11/25/2017 22:50	5.255.250.65	United States	Not Logged In
11	1918	authors	https://healingmentaldisorders.com/wp-content/uploads/2017/08/healingmentaldisorders_com_authors.pdf	11/25/2017 22:03	100.43.81.117	United States	Not Logged In
10	1919	cause	https://healingmentaldisorders.com/wp-content/uploads/2017/08/healingmentaldisorders_com_cause.pdf	11/25/2017 21:36	100.43.81.117	United States	Not Logged In
9	1926	healers	https://healingmentaldisorders.com/wp-content/uploads/2017/08/healingmentaldisorders_com_healers.pdf	11/25/2017 21:23	100.43.81.117	United States	Not Logged In
8	1931	opinions-2	https://healingmentaldisorders.com/wp-content/uploads/2017/09/healingmentaldisorders_com_opinions_2.pdf	11/25/2017 18:30	93.158.161.112	Russian Federation	Not Logged In
7	1916	addiction	https://healingmentaldisorders.com/wp-content/uploads/2017/08/healingmentaldisorders_com_addiction.pdf	11/25/2017 18:07	5.255.250.5	United States	Not Logged In
6	1924	FAQ-3	https://healingmentaldisorders.com/wp-content/uploads/2017/11/healingmentaldisorders_com_faq_3.pdf	11/25/2017 17:35	93.158.161.67	Russian Federation	Not Logged In
5	1922	FAQ-1	https://healingmentaldisorders.com/wp-content/uploads/2017/08/healingmentaldisorders_com_faq.pdf	11/25/2017 16:51	5.255.250.144	United States	Not Logged In
4	1932	patient	https://healingmentaldisorders.com/wp-content/uploads/2017/08/healingmentaldisorders_com_patient.pdf	11/23/2017 19:37	157.55.39.232	United States	Not Logged In
3	1931	opinions-2	https://healingmentaldisorders.com/wp-content/uploads/2017/09/healingmentaldisorders_com_opinions_2.pdf	11/23/2017 19:37	207.46.13.83	United States	Not Logged In
2	1928	mantras	https://healingmentaldisorders.com/wp-content/uploads/2017/08/healingmentaldisorders_com_mantras.pdf	11/23/2017 19:32	207.46.13.41	United States	Not Logged In
1	1933	sleep travels	https://healingmentaldisorders.com/wp-content/uploads/2017/08/healingmentaldisorders_com_sleep.pdf	11/23/2017 19:32	207.46.13.41	United States	Not Logged In

Why on Earth my reply was put “for moderation by our automated system. It will be reviewed within 72 hours.” Can this restriction be removed? I didn’t do anything wrong.

Deep breath: it was all the links in your log. I’ve approved it in the pending queue.

Being caught in the spam queue isn’t about doing something wrong. I’ve had my posts here flagged as pending too and I’m a moderator here. I don’t try to understand the automated spam system. ??

@timofeia, thank you for providing the list of IP address. This will help the developers investigate further your issue.

Kind regards

It will take a little time to perfect this feature as we get reports and add new user-agents to check in that function. I have just updated the plugin that should help. Let me know how it goes after you upgrade the plugin.

Also, I have added a filter in there so you can write your custom function to check for bots if you want to.

Thank you. I upgraded the plugin today (to 3.5.0). I’ll check how it goes.

Many bots are still active (180.76.15.25 – China, etc). Maybe we can use:
if (preg_match(‘/bot|crawl|curl|dataprovider|search|get|spider|find|java|majesticsEO|google|yahoo|teoma|contaxe|yandex|libwww-perl|facebookexternalhit/i’, $_SERVER[‘HTTP_USER_AGENT’])) {
// is bot
}

• This reply was modified 7 years, 3 months ago by timofeia.

Can you tell me what the user-agent value of this bot is? I can the add it’s signature to the list.

User-agent value for this bot is: Baiduspider

Thank you. I have added it to the plugin’s code.

Please do the following to test it on your site so we can see if there is anymore that need to be added before I release a new version of the plugin.

1) Open the following file of the plugin:
simple-download-monitor/includes/sdm-utility-functions.php

2) Search for the following function in that file:
sdm_visitor_is_bot()

3) Add the ‘baiduspider’ entry in the $bots array.

Let me know if that takes of everything on your site.

I added the ‘baiduspider’ entry in the $bots array. I’ll check in the next 24hrs. Yesterday I had also from 164.132.162.164 (France bot), 217.182.132.193, 217.182.132.183, 217.182.132.70 (UK/France bot).

After I used the array below, I did not have any bots logging for the last 24 hrs:

$bots = array(‘archiver’, ‘Baiduspider’, ‘Bingbot’, ‘binlar’, ‘casper’, ‘checkprivacy’, ‘clshttp’, ‘cmsworldmap’, ‘comodo’, ‘curl’, ‘diavol’, ‘dotbot’, ‘DuckDuckBot’, ’email’, ‘Exabot’, ‘extract’, ‘facebot’, ‘feedfinder’, ‘flicky’, ‘googlebot’,’grab’, ‘harvest’, ‘httrack’, ‘ia_archiver’, ‘jakarta’, ‘kmccrew’, ‘libwww’, ‘loader’, ‘MJ12bot’, ‘miner’, ‘msnbot’, ‘nikto’, ‘nutch’, ‘planetwork’, ‘purebot’, ‘pycurl’, ‘python’, ‘scan’, ‘skygrid’, ‘Slurp’, ‘sucker’, ‘turnit’, ‘vikspider’, ‘wget’, ‘winhttp’, ‘yahoo’, ‘YandexBot’, ‘youda’, ‘zmeu’, ‘zune’);

Thank you very much. I will add it to the core plugin and it will go out with the next release.