I’m having the same problem now. I migrated to Yoast SEO from All in one SEO 4 days before. Before Yoast XML, I used another plugin called Google XML sitemap. After activating yoast sitemap, i de-activated and deleted the other sitemap plugin.
The Yoast XML submits the sitemap without any error but it doesn’t get indexed fully. My recent posts also won’t show up in google search results.
I hope it gets indexed fully after some time, just like your’s.
How is Yoast XML working now? Is it good enough or should I go back and use Google XML sitemap plugin?