Why Is It…

Hi @deeveearr

Sorry to hear you are having this issue.

Some developers say that a sitemap is not necessary for one page sites, I do suggest creating a simple sitemap with a home URL only, indeed, I extend saying that nowadays with GDPR and similar laws there is no one site anymore, it is important to have a term page, but it is a topic for another discussion.

I tried to replicate this on my lab site this behaviour but couldn’t replicate it, but we found a bug on the sitemap missing when there is only one page with the front page set to it, and we reported it to our developers.

By 2 pages, was it a trashed page or non-existing page?

About characters, the correct is 160 and this is going to be updated in SmartCrawl in the upcoming version.

I hope you can give a new try in SmartCrawl and feel free to contact us any time you need.

Best Regards
Patrick Freitas

Hi @wpmudevsupport12

I’m back on Yoast at present, but as far as I can remember:

Sitemap One didn’t mention the page at all – it just mentioned an elementor template.

Sitemap Two (inside sitemap one) was again an elementor template, but a different one to the one in sitemap one.

Oh, and no pages have been trashed.

Yoast can create a sitemap, no problem – even if a page had been previously trashed, as I’ve got Yoast running on many other websites – I’m just not struck on the reputation of their new owners.

I’ll try smartcrawl again when it’s been updated.

Hi, @wpmudevsupport12

Between the last post and this one, I changed over to The SEO Framework instead of Yoast, TSEOF creating sitemaps easily.

On your insistence that there are no One-Page websites anymore, I also created two extra pages – Terms and Privacy – and added those to my 1-pager.

TSEOF found the two new pages immediately and threw them into the new sitemap.

However, the new version of Smartcrawl (released 6 hours ago at the time of writing) still cannot get its head around sitemaps and still publishes Elementor Templates, even though the two new pages are NOT built with Elementor.

I didn’t even bother checking whether the word-count had been updated as the sitemaps were my main priority.

In the interests of keeping my Google rankings therefore, I have disabled and deleted Smartcrawl.

Howdy! TSF developer here. I got pinged because you mentioned The SEO Framework. I’m here to rant about Yoast SEO and Elementor in defense of SmartCrawl.

My rant here is for the record. None of the below is meant to undermine your findings or opinion. Let’s get to it.

Sitemaps barely work besides the one from Yoast SEO because they spread misinformation, causing vendor lock-in. I explain it here: https://theseoframework.com/blog/an-introduction-to-a-thousand-changes/#features-sitemap.

To our disgrace, we found that hosting parties (Namecheap, EasyWP, SpinupWP, to name a few) do not know much about NGINX. Instead of configuring their servers to work with WordPress correctly out of the box, they blindly implemented vendor-specific code from Team Yoast to output sitemaps. These rules forbade all other sitemap implementations but the ones from Yoast SEO.

Elementor’s templates are also programmed incorrectly: They’re marked as public Post Types while a) they shouldn’t be Post Types, and b) they aren’t meant for the public. They uselessly maintain a compatibility file specifically for the larger plugins patching their error — this to the detriment of everyone else.

None of these issues is Smartcrawl’s fault. However, unfortunately, we (other (SEO) plugin developers) are still left to fix them, whilst the big boys from Team Yoast and Elementor are stoically making developing for WordPress a nightmare.

Hi @cybr

Thanks for the clarification on Yoast and Elementor.

I’ve already mentioned that I’m looking to move all of my SEO plugins away from Yoast, as I don’t like their new owners’ reputation.

I also find that Elementor needs to be disabled and deleted, then re-installed and re-activated on most of their upgrades, so the two websites that I have using Elementor at present will not be reigning for long.

My main money site is running AIOSEO at present and I’m looking to place another of my money sites onto that soon.

TSEOF is really fiddly from what I’ve sampled so far on a 1-pager, but it gets the job done!

From what I’ve seen of Smartcrawl though, it cannot be considered while their sitemaps are pulling in Elementor post-types, whether or not they’re supposed to be public or private.

Hi @deeveearr

Thank you for your responses!

I won’t discuss other SEO plugins here but taking a step back, let me refer to the specific issues that you reported in context of SmartCrawl.

1. Elementor templates

@cybr is right about the Elementor templates – they are for some reason created as “public” custom posts. Neither SmartCrawl nor other SEO plugin can “guess” by “something” (name or other attributes) whether a given post type should or should not be public. Nothing stands in a way for somebody to create a custom post type that would be very same in configuration as Elementor’s template but on the site that does not even use Elementor and for a completely different purpose.

The only way to decide whether to include posts of such type in sitemap or not is to “inform” SEO plugin about it. Agreed, it’s quite possible that some plugins have that “built-in” (meaning – some developer just specified in code that given post type should not be included; maybe even adding additional check to see if Elementor is active on site or not). But that’s just one way of doing that and not particularly “reliable” in a long run.

In SmartCrawl we took a different direction and we let you decide.

It’s as easy as going to the “SmartCrawl -> Sitemaps” page -> “General Sitemap” tab and switching off all the post types that you don’t want to be included in sitemap. You’ll se a set of toggle switches for that in “include” section.

If a given post type is not listed there – it wouldn’t be SmartCrawl issue but the issue in the code/plugin that adds/creates such posts type. But that’d be rare case and Elementor’s post types would be listed there so you can simply switch them off.

2. Sitemaps

SmartCrawl by default and design creates “split” sitemap, meaning there is a main sitemap.xml file and that file includes links to “sub-sitemaps”. Those files are “organized” based on post types/taxonomies, plus there may in some cases be “extras” sitemap (this one shows up only if there are some URLs specifically, manually added to plugin – to be included even though they were not detected for some reason).

Those sub-sitems do have numbers in filenames (like e.g. page-sitemap1.xml) as there’s certain limit of URLs that can be included in a single sitemap. So if there’s more URLs there’d be insted page-sitemap1.xml, page-sitemap2.xml (and so on) files.

This is perfectly fine for Google and other search engines. It does make no difference for them if the sitemap is just one file or 50 files, with different names. They don’t even have the “sitemap” word in file name – for as long as they are correctly structured and correctly submitted to Google.

Now if it comes to the “one pager” and where the URL of it is or is not, that’s a bit different story and I can’t tell much more without actually taking a look at the site itself and the sitemap generated by SmartCrawl.

By default, if it’s a “one pager” and it does have a page set as “homepage” in WordPress settings, that should be included in “page-sitemap1.xml” sitemap (linked from main “sitemap.xml”). But if the configuration of the site is different than this there may be some additional factor in play.

If you do have some copy of the site (or would be able to set it as it was when you reported the issue) we can take a look at it to see what more/else may be involved.

Kind regards,
Adam

I thought that I’d give Smartcrawl another run out.

This time the sitemaps needed troubleshooting, and just included a link back to this forum, as clicking on the sitemap link just said something like – ‘this page has not been created yet’.

Had enough of it now – and also the meta description is still set to 300 characters.

Deactivated, deleted and thrown in a bin that I found five miles away.

Hi @deeveearr,

Sorry to hear about that. It’s tough to say what exactly you noticed without seeing the sitemap.xml page live.

Since you have removed the plugin, what we could suggest is limited. In case you want us to check this further, maybe you could share a screenshot of what exactly you notice on your side and also the site URL preferably a staging URL. Then it would be helpful.

Kind Regards,
Nithin

Hi @wpmudevsupport11

No, it’s ok.

I tried creating a staging site before, for another SEO plugin that was having trouble creating sitemaps, and it threw up loads of indexing errors in Google.

I think we’ll call it a day.

Hi @deeveearr

Thank you for the update, to prevent the pages from being indexed you can block the robots.txt https://wpmudev.com/docs/wpmu-dev-plugins/smartcrawl/#robots-txt-editor

But in case you decide to troubleshoot this in the future feel free to let us know.

Best Regards
Patrick Freitas

Right, I’ve put Smartcrawl onto my old staging site for one of my main money sites.

First things first before I even look at the sitemaps, there is an issue:

Please add at least one recipient to enable the scheduled report.

What’s all that about?

Hi @deeveearr

This is related to the reporting feature. SmartCrawl can perform regular, scheduled scans (crawls) of site to compare them with sitemaps and report to you e.g. if it found some URLs on site that are missing from sitemap. The same way it can also run regular SEO Health/audit tests and report them via e-mail

The message you get is just an information that such reporting was enabled but never configured on “SmartCrawl -> Sitemaps -> Reporting” page and/or on “SmartCrawl -> SEO Health -> REporting” page.

You are not “required” to use any of those functions, you can just disable them.

However, the fact that you do have them enabled would mean that you are using Pro version of the plugin.

Note please that we cannot provide any more support for Pro versions here as it’s against the rules of this support forum.

If you need any further help with Pro version of the plugin, please reach out for support on our site here

wpmudev.com

Kind regards,
Adam

Huh?

Hi @deeveearr

Huh?

You wrote you are getting this message from SmartCrawl: “Please add at least one recipient to enable the scheduled report.”

This message relates to “Reporting” feature of SmartCrawl which is not available in free version – only in Pro. The message suggest that reporting option (either for Sitemaps or for SEO Health tool, or both) is enabled but not fully configured. It can’t be enabled in free version of SmartCrawl because it’s available in Pro only.

IF I’m missing something here, please feel free to correct me if I’m wrong.

Kind regards,
Adam

However, the fact that you do have them enabled would mean that you are using Pro version of the plugin.

Why on earth would I even consider using the pro version?

So now, we have someone who cannot enable sitemaps due to them being perpetually being created wrongly, but it’s my fault as I’m using the pro version?

I’ve never heard so much rubbish from a developer or customer service rep.

Smartcrawl will now be relegated back to the bin and I’ll have a refund please.