Request: Log filtering

Glad to see you again, Tony G, and thank you for requesting!

Yeah, that’s good idea!!

• To find requests that should be passed, use

• [?] Display Whitelisted and [?] Display Blocked
• [?] Display Blacklisted and [?] Display Blocked

• To find requests that should be blocked, use

• [?] Display Whitelisted and [?] Display Passed
• [?] Display Blacklisted and [?] Display Passed

I think a hook for log entry is the easiest way to mark something like ? or ? which can be handled by the current “Search in logs” filter.

Please kindly give me some time to consider how to do that.

Thanks again!

• This reply was modified 6 years, 3 months ago by tokkonopapa.

Hi Tony G,

I made an alpha version of 3.0.14 at GitHub where a new filter hook ip-geo-block-logs is available.

Here’s a screenshot:

Currently, there’s no new check box but just we can copy & paste some special characters like 1, 2 and 3 into “Search in logs” text box.

• 11: Passed and Whitelisted
• 12: Passed and Blacklisted
• 13: Passed and Not listed
• 21: Blocked and Whitelisted
• 22: Blocked and Blacklisted
• 23: Blocked and Not listed

I hope you to try and check the functionality.

How to update:

1. Download 3.0.14a zip file and unzip it. You can find ip-geo-block folder.
2. Upload ip-geo-block into your plugins directory. Overwriting the existing one is OK.
3. Download a zip file “drop-in-admin.php” from Gist. Unzip it and then upload it into the directory of Geolocation API library which is typically placed at /wp-content/ip-geo-api/.
4. Done!

Actually it’s useful for me when I want to optimize the validation rules such as “UA string and qualification“. But I can not decide whether or not to put the proposed UI (e.g. check boxes) into the upcoming 3.0.14 core function, because I think only us geeks can understand it, isn’t it?

Any feedback will be appreciated.

Thanks.

@tokkonopapa – I’m sorry I missed your prompt response. Sincere thanks as always!

I am just seeing this now but I will install the alpha test within the next day or so. I frequently check the logs to see what new connections are coming in, and I am always missing this concept of filtering.

For example:
– First filter everything that was blocked. Verify to make sure I’m not over-blocking.
– Now filter just the passed requests, and all IP’s beginning with 00.00. Delete the logs, I know they’re OK.
– Now re-filter just the 11.11 IPs. Sort by the query. All valid? Delete the logs.
– Now filter everyone from TR. Show all 212 records on the same page. Block all of those IPs for now. (not the countries)
– Now show the blocklist for all IPs that are in CZ, delete them from the list, then block the country.

As you can see, there is a work flow there, and it changes every time I look at the log. I would be happy to use hooks or some other mechanism so that I can define filters and sorting, and then take action on whatever is currently displayed.

With some info about the log schema, what about a textbox to enter a custom query? You can display the results of the query, maybe a max of 1000 records, it will be our responsibility to LIMIT the result set to a reasonable number. Then the controls are already there for acting on checked items.

Regards.

Hi Tony,

I’m pleased to know that this plugin is a bit helpful for the workflow of managing your sites.

I’m almost ready to release a new version. So you don’t need to test alpha version.

And thank you for the details about your workflow. I’ll try to understand one by one ^^)

– First filter everything that was blocked. Verify to make sure I’m not over-blocking.

This can be done by sorting by “Result“, but I think you need easier way, right?

– Now filter just the passed requests, and all IP’s beginning with 00.00. Delete the logs, I know they’re OK.

This should be done by passed 00.00 in “Search in logs” text box. A space is considered as “and”.

– Now re-filter just the 11.11 IPs. Sort by the query. All valid? Delete the logs.

How do you do “Sort by the query”? What “query” do you need?

– Now filter everyone from TR. Show all 212 records on the same page. Block all of those IPs for now. (not the countries)

You can do this by bulk action “Add IP address to Blacklist“, right?

– Now show the blocklist for all IPs that are in CZ, delete them from the list, then block the country.

I think this is tough work if there are many IPs in blacklist.

what about a textbox to enter a custom query?

I think the query you need is something like this: “select data which has 00.00 in IP address”, right? I think the same thing can be done by “filter” which should be fully customizable.

The new version 3.0.14 has ip-geo-block-logs filter hook, but I know that’s not enough for you to speed up your workflow. So I’ll keep thinking about “custom filter” for better solution, and propose you.

Please keep discussion!

Thanks.

P.S. https://datatables.net/reference/api/search() is the document related to “Search in logs” text box. The search function can handle the regular expression, but it’s not available in this plugin.

• This reply was modified 6 years, 2 months ago by tokkonopapa.

I will try to clarify my notes above…

– First filter everything that was blocked. Verify to make sure I’m not over-blocking.

This can be done by sorting by “Result“, but I think you need easier way, right?

I’m learning more about the Search In Logs box as we discuss this. It searches all available text in the logs. So to filter only Result=passed logs, just enter “passed” in the box. OK, this one is done!

– Now filter just the passed requests, and all IP’s beginning with 00.00. Delete the logs, I know they’re OK.

This should be done by passed 00.00 in “Search in logs” text box. A space is considered as “and”.

When I saw your note there I suddenly understood how that box works. OK, I understand this now!

– Now re-filter just the 11.11 IPs. Sort by the query. All valid? Delete the logs.

How do you do “Sort by the query”? What “query” do you need?

I meant “sort the logs that are now filtered”, so just click the column header. No action required, this is just describing the flow…

– Now filter everyone from TR. Show all 212 records on the same page. Block all of those IPs for now. (not the countries)

You can do this by bulk action “Add IP address to Blacklist“, right?

We cannot change the number of records displayed in the UI. It’s hard-coded to 10. Rather than paging through the list I was hoping we could use a dropdown to display 50, 100, 500, or 1000 logs.

If we click the header checkbox, it selects the 10 entries currently displayed. Select any action and then Apply. It removes logs, or adds IPs to the list, only for all IPs that are currently displayed. Not the subset of all IPs that are currently filtered.

For example, search for “gecko” to select only visitors who use Firefox. Check the header, to select All records. Select ‘Add Address to a list’, and click Apply. Only the IPs from the first page are saved to the list. To ensure we save all IPs for some selection, we need to go through every page.

– Now show the blocklist for all IPs that are in CZ, delete them from the list, then block the country.

I think this is tough work if there are many IPs in blacklist.

Yes, but the idea is that if we’re blocking an entire country then we no longer need to have all of the IP addresses for that country in the blocklist. If we do not remove all of those items then they are all processed on every new connection, which slowly degrades performance.

what about a textbox to enter a custom query?

I think the query you need is something like this: “select data which has 00.00 in IP address”, right? I think the same thing can be done by “filter” which should be fully customizable.

You are correct. Now that I understand how the search box works it will be much easier to filter records.

I think we have covered everything here so far.

You referenced the DataTables plugin for jQuery. What do you think about adding a box for regexp queries? Or, if the first character of the search is ^ then treat it like a regexp?

I have installed drop-in-admin.php and I think it’s a great solution for now. ??
Thanks!!!!

• This reply was modified 6 years, 2 months ago by Tony G.

Hi Tony,

I have installed drop-in-admin.php and I think it’s a great solution for now.

That’s great! And you may need UI to define your custom filters.

Rather than paging through the list I was hoping we could use a dropdown to display 50, 100, 500, or 1000 logs.

jQuery DataTables have this functionality. So I’ll achieve this. Then you will be able to select many IPs and to execute bulk action.

And I also consider whether we can select mutiple IPs accross the pages or not.

Or, if the first character of the search is ^ then treat it like a regexp?

^ is one of the special charactor that means “top of the word” in regular expression. So for example /regexp/, %regexp%, !regexp! or whatever may be better.

OK, I have a lot of things in my to do list. I’ll try step by step.

Thanks always!

Dear Tony and all,

I released the new version 3.0.15. The preset filters can be available in it.

Please try new filter hooks ip-geo-block-logs[-preset] and drop-in-admin.php to make your workflow easy.

Thanks.