Google crawling pages that don't exist

I know this has been posted before but I couldn’t apply the solution to my problem.
Google keeps generating crawl errors (404) for content that no longer exists and when I check where it’s being linked from, it says its coming from my website. My website is not that old and I don’t have that much content so I am 100% sure it has been removed.
The only thing I can think of is it may have something to do with Yoast’s Seo plugin since some of the links Google shows points to my website’s sitemap.

Any help would be greatly appreciated.