Created a new page with just one pattern and there are no issues.
I previously deactivated Gutenverse (on a dev server) and the problem went away, that was how I narrowed it down to the patterns. The problem with what you are suggesting is that if I do that my site is unusable to the outside world and would take to long to fix all of the issues before reenabling Gutenverse. At present the site works for the user, I just can’t edit any of the previously created pages. If disabling Gutenverse didn’t break the theme and layout I would do that. This all worked as expected until an update that changed the ability to edit patterns on the page they were used on.
After further searching it looks like I am experiencing something similar to this post