I went in and updated the output.php of the plugin to not output for non amp pages.
I basically added this code
if( ! (function_exists( 'ampforwp_is_amp_endpoint' ) && ampforwp_is_amp_endpoint()) ){
return ;
}
This has stopped the problem for the time being where my FAQ was getting duplicated etc.
But need a better fix for long term.