Hook to load CPT templates from Plugin directory?

I am building a theme that creates various CPTs (GeoDirectory).

I am making various single.php templates for the CPTs.

I would like to place those templates in a custom plugin, instead of using a child theme. That way, my CPT templates are more portable in case I change themes.

Is there a way I can use a hook to load CPT templates from a different directory for specific CPTs?