Hi Aaron,
Sitemaps don’t need to include every URL all the time. The sitemap doesn’t contribute to ranking. For Google News, sitemaps may only contain 1000 links at a time, yet all news indexed by search engines for a long time–if not permanently.
Once a URL has been spotted by a search engine (from the sitemap or anywhere else), search engines will store the URL in their processing queue. They reprocess the URL periodically until it’s no longer available or marked with “noindex” for some time.
Now, the sitemap can help speed up indexing–nowadays, since the advent of structured data, that’s its only purpose. This is where The SEO Framework shines: it orders the entries of posts so that the ones updated most recently are displayed on top. In contrast, pages are displayed in chronological order of publishing–but sites typically only have a few pages compared to posts.
In your case, products should be treated as posts.
Via the <lastmod> tag, the sitemap notifies search engines of a recent change.
With all that said, I want to suggest lowering the number of URLs queried by the sitemap.
TSF can save your sitemap via transient caching. You can find this option under “General > Performance”, which will allow storing a cached “semi-static” copy of the sitemap in your database. The cache will flush whenever you update any vital part of your site, or when you update or publish a post, page, or product. You can instigate its caching via the command line by touching the sitemap via wget or cURL–but, this might be detrimental to your server’s resources as Bing and Google try to fetch it soon after they’re pinged.
Paginated sitemaps are on the roadmap for The SEO Framework, but it’s receiving such a low priority it’s not worth mentioning any further. So, if you still want to enjoy paginated sitemaps, then you can use a different plugin to replace The SEO Framework’s sitemap functionality.
For example, the Google XML Sitemaps plugin. Please do keep in mind that errors might pop up in Google Search Console, as it doesn’t recognize the indexing behavior of The SEO Framework.
I hope this covers all the bases of your inquiry! Cheers ??
