Google News – “There are no articles in this section”

Hi Smchugh,

Thank you for letting me know about this. I spent last night coding a ‘tester’ to mimic the feed being fetched by the Publisher Center. When I fetch your feed using that, I’m getting an internal server error.

I’m working on identifying what’s causing that and will have a fix out asap. Are you comfortable with looking at your error logs to see if you see an error related to https://www.jockrock.org/category/news/feed/gn ? If you don’t know how to do that, don’t worry about it,

Chris

Opps, spoke to early, it looks like your feed is inaccessible.

The Publisher Center uses a crawler called ‘FeedFetcher’ and it appears that FeedFetcher is being blocked by a firewall of some kind.

Are you using any kind of ‘bad bot’ plugins that might be blocking it?

It might be related to CloudFlare, I’m getting a lot of similar reports from others using CloudFlare…. but I use CloudFlare myself and it’s not blocking my feed. I’m getting in touch with CloudFlare support though to see if they have any thoughts on this,

Chris

Smchugh,

Could you take a look at something for me?

If you log into your cloudflare account, then go to ‘firewall’ and then click on ‘add filter’ and set it up to look for ‘user agent’ equals:

FeedFetcher-Google; (+https://www.google.com/feedfetcher.html

and then click ‘apply, does that display any blocks of feedfetcher?

Chris

• This reply was modified 4 years, 6 months ago by christopheran.

Hi Chris, thanks for looking into this. I set up the Cloudflare filter as you suggested, and got “No firewall events found matching your filters”

However, I’ve got at least one Bad Bots-style plugin installed, no idea if it’d be blocking it but since you mentioned that I’ll look and see if that seems possible as well.
Edit: disabled Stop Bad Bots but no difference made
Thanks
S

• This reply was modified 4 years, 6 months ago by smchugh.

Answering these in reverse order Chris – here’s what may be useful error log info from debug.log. There’s 22 or so of these sections, possibly coinciding with every time I tried to refresh the articles in GN…

05/22/20 19:17:36.654 [76.106.143.24:41054 1 yGl] ?? ——GET HTTP/1.1 (HTTPS) /category/news/feed/gn
05/22/20 19:17:36.654 [76.106.143.24:41054 1 yGl] Query String:
05/22/20 19:17:36.868 [76.106.143.24:41054 1 yGl] [Router] get_uid: 0
05/22/20 19:17:36.868 [76.106.143.24:41054 1 yGl] [Router] get_role:
05/22/20 19:17:36.869 [76.106.143.24:41054 1 yGl] GUI init
05/22/20 19:17:36.967 [76.106.143.24:41054 1 yGl] [Ctrl] not cacheable before ctrl finalize
05/22/20 19:17:36.968 [76.106.143.24:41054 1 yGl] [Router] get_role:
05/22/20 19:17:36.968 [76.106.143.24:41054 1 yGl] [Vary] role id: failed, guest
05/22/20 19:17:36.968 [76.106.143.24:41054 1 yGl] [Router] get_role:
05/22/20 19:17:36.968 [76.106.143.24:41054 1 yGl] [Router] get_role:
05/22/20 19:17:36.969 [76.106.143.24:41054 1 yGl] X-LiteSpeed-Cache-Control: no-cache
05/22/20 19:17:36.969 [76.106.143.24:41054 1 yGl] [Optm] bypass: Not frontend HTML type
05/22/20 19:17:36.969 [76.106.143.24:41054 1 yGl] End response

Ha, well ?? the 76.106.143.24 is my ip address, that’s me banging on the door and trying different things to see what’s happening.

Ok, will have to go back to the drawing board and figure out another way of testing why Google’s not able to get those articles,

Chris

Thanks Chris, any suggestions welcomed! FYI, I tried turning off all plugins (including Bad Behaviour, another bot blocker) and running the GN scan for articles again, but no change I’m afraid.

Ok, hang on, I’m still investigating –

Hi Smchugh,

Thank you for your patience – I think I’ve got a fix in place for this now, can you update to version 1.0.8 and let me know, I’ll test and we’ll see where we are,

Chris

Hey Chris – that seems to have done the trick! The articles are loading certainly, I guess I just need to wait to see if Google are happy with it, But thanks for all your help, hope sorting my problem out will also fix issues for others.

Cheers – Stuart

Yes, it has helped with others also, it always helps to have more than one report so things can be narrowed down.

Are you using a plugin that ads a featured image to your rss feed? I see one in the feed using the JetPack/WP CDN (i2.wp.com). The Publisher Center won’t accept that because it’s on a different domain than yours… ok, I should say they used to not, I’m not sure if they have whitelisted i2.wp.com, I’ve asked them to.

Anyway, please try the GN Publisher URL in the Publisher Center – and be aware that it won’t immediately change articles you’ve already uploaded to the Publisher Center – only new or updated ones. and let’s see if everything renders ok, or if there is a doubled image issue. If you are using a plugin that adds a featured image to the feed, you might need to turn that off if it doesn’t affect something else (like you are using your default feed for something else),

Chris

Very interesting… I installed a plugin, Featured Image From URL, but not onto Jockrock.org but my other site, isthismusic.com. (Jockrock doesn’t need it, fortunately). I do take a feed from that onto the front page so it could be that some data crosses over and you can see it?

I’m getting double images so have unchecked the box, the Refresh on GN is really slow so hopefully that will sort it – I’ll check tomorrow, it’s midnight now. But I suspect that Google won’t authorise it in its current state, I need to sort out new graphics (now I can actually preview I can see these don’t work well) plus there seems to be a ‘Subtitle’ field I could do with filling somehow, otherwise I’ll need to completely adjust how I write my headlines. I may be best withdrawing the site from the process for now but I’ll leave it for now so I can get you any more info you need for the plugin.

Yeh, the refresh in the Publisher Center seems slow, but it’s odd too, if you track your logs, they will often crawl the feed right after you click the refresh, but they don’t show the ‘updated’ time unless there was a change in an article, sometimes… other times, if it’s been a while, it does display the updated time faster.

Here’s a trick you can do with GN Publisher though – any time an article is updated, it will ping Google to let them know there’s a change in that article and they will update that article. So… for example, we want Google to reload the article where you ended up with a doubled image. Just edit that article and, I don’t know, just add a space, or make some other small change, and save it. GN Publisher will ping Google, Google will update the article in the Publisher Center – be sure to refresh the entire Publisher Center page with your browser’s refresh button after a few minutes, then recheck. I just checked your feed and the image GN Publisher adds is gone, it’s just the i2.wp.com image now, so if the Publisher Center is picking that up ok, it should just display one (in the article),

Chris

Cheers Chris – am getting there slowly but surely! Fully refreshing the page does seem to be essential (I’d already realised that needing to add a space or whatever to a setting and resaving was another oddity).

I have got the subtitle working much to my surprise, by using the ‘Secondary Title’ plugin. Just trying to suss out whether I should be checking ‘Include featured image’ on GN Publisher as I think Google is getting confused at the reloads (I edited/resaved all the existing posts). Brand new post has the duplicate image. I think I need to generate a bunch of new posts just to test this out! Thanks – S

Ok, since we have original issue resolved, I’ll go ahead and mark this resolved, but I’ll still be here and you can still add to this thread as things progress.

It is hard to overcome Google cache of each article, posting new articles to see what appears generally works ??

Chris