Interesting product but not mature (inconsistent scan)

Hi @aberbenni,

When Cookiebot scans your website for online trackers, you recieve a list of all URLs that Cookiebot found. There is no such thing as unknown links. All links are available in the scan report, and they are only in the scan report, because the URLS were publicly available at the time of the scan.

If you believe there is an error, you are welcome to reach out at wpsupport@cybot.com and we can look into it.

I sent a contact request on 25/05/2018 still waiting for reply…
I sent a new one today.

@aberbenni

We do apologize for not replying faster.

Can you send me some more information on wpsupport@cybot.com on which ticket I should look for?

I sent you copies of the two support requests.

Reading your WordPress help page https://support.cookiebot.com/hc/en-us/articles/360003797213 I have to say that there are no image pages in website.
But, I saw that Cookiebot, in the second scan, counted every tag page, category page and blog index page (eg /blog/2 /blog/3 etc) even if this is not reported in your help page (linked above).

@aberbenni

Unfortunately I did not receive anything on wpsupport@cybot.com from you. There is nothing in the spam folder either.

Although it may be hard to understand why such “dumb” pages are counted, they are publicly accessible, and can set different kinds of trackers, which is why they are in the page count.

See also: https://www.ads-software.com/support/topic/not-interesting-for-bloggers/

I sent a new mail to wpsupport. Ticket is #2652.

I’m focusing more on the 192 extra pages, found on second scan.
174 (of those 192) are wordpress tag pages, 5 category pages, and some other pages undiscovered at first scan.
Considering an unmodified site, how could this be possible?

After reading https://www.ads-software.com/support/topic/not-interesting-for-bloggers/ I have to say that first scan was only partial. Maybe something went wrong during scan.

@aberbenni

I’ve checked your tickets, and the “unknown links” you are referring to.

It is true that those URLs do not exist on your webpage at the moment, however they were active at the time of our scan, and did return a 200 OK status code, otherwise we would not have included them.

By inspecting your website, I noticed you have a hidden container, referring to something called linkexchangefree. This is probably for SEO link building.

<div style="display:none">
			<a href="https://yourdomain/links/tl.php">Resources</a> - <a href="https://www.telalinks.com/" target="_blank">Free Link Exchange</a>
			<a href="https://www.linkexchangefree.com" title="Link Exchange Free">Link Exchange Free</a> :: <a href="https://yourdomain/links/ef.php" title="give one link take 10000+ link of your website on other website in less than 5 minutes">1000+ Links Free</a>

I don’t know where you have this code from, but your use of linkexchangefree is generating these so-called unknown URLs.

Also, if you go to The Wayback Machine and enter one of those “unknown links”, you’ll see that they were once active, so Cookiebot did it’s job, and it did it well.

• This reply was modified 6 years, 9 months ago by cookiebot.

What is probably going on, is that linkexchangefree is generating backlinks to other websites from your site, and the same is most likely happening the other way around. This may occour when linkexchangefree detects that your website is being crawled.

Thanks for your deep analysis. But how could you explain those 192 extra pages, found on second scan (18/06/2018), not found on first scan at 25/05/2018?

Did you change scan policy in the meantime?

@aberbenni

As you see, when we check your site today, those unknown URLs are not available.

When we checked the first time, there were X number of them available, and when we checked the second time, there were Y number of them available.

This means that something is going on dynamically on your backend.

Either linkexchangefree is adding random URLs in random intervals, or you did something to stop it from doing so, after we found the “backlink pages”.

Please reply to my second question, this is unrelated to the first.
I think this is most important because adding pages from one scan to another could change subscription level and price.

Using The Wayback Machine you can verify that the wordpress tags and categories where there on 25/05/2018 but unrevealed by scan why?

Tags and categories have nothing to do with linkexchangefree, they are WP managed.

174 wordpress tag pages, 5 category pages, and some other pages undiscovered at first scan where discovered on 18/06/2018, why?

@aberbenni

We have not changed our scan policy recently. Let us try to clarify about the number of subpages identified:

If you are on a free subscription, up to 100 pages are being scanned.

If you have been moved to a free 1-month trial because your domain had more than the allowed 100 subpages, then the scan report and the cookies identified will be based on (up to) ~200 pages. This is the number of pages that the scanner analyzed when it determined that your site had more than 100 pages. If you have more than 200 pages on your website and would like a full scan of your entire website, then you should upgrade to a premium subscription.

If you would like to know how many subpages your domain has, you can order a free quote from https://www.cookiebot.com/goto/quote-input/. Attached to this quote is an URL list of up to 5,000 URLs identified in order to determine the subscription size and price.

So, if you have ordered a price quote (or a compliance test from the front page of https://www.cookiebot.com where you have requested a price quote to be included in the report) then the URL list will show the total number of subpages identified on your domain (up to 5,000).

On the 25th may we scanned 201 pages on your website, because you were on a free-trial.

On the 18th june you requested a compliance scan including a quote. Because you included a quote, you got a full scan of your site.

On the first scan, the “unknown” links were available, and a part of the 201 pages we scanned.

On the second scan, the “unknown” links were no longer availabe, however we did a full scan this time, where we found all other pages.

We hope it is clear now that this is not caused by an immature product/scanner. Also, we have updated this article to try and make it even more clear for our users:

https://support.cookiebot.com/hc/en-us/articles/360004612233-What-is-a-free-trial-and-when-will-I-pay-the-subscription-fee-

Ok so the scanner works. Thanks for clarification. Scan limits weren’t obviuos at that time.

I think you need to alert users of this at scan/subscribe time and not hide this info only in a support page.

@aberbenni

We inform about it in the mail you receive, when being upgraded to free trial, including links to different knowledge-base articles.

Since you have missed it, we will consider how we can make it more clear.

Thanks for your input.