Amp Browser Cache

By browser caching I assume that means adding Cache-Control headers for assets: https://www.wpfastestcache.com/optimization/leverage-browser-caching/

This won’t apply for CSS files because all stylesheets are inlined. It also won’t apply for JS files because they are not allowed in AMP. It would be relevant for images, however.

So it’s no problem, but it will be of limited benefit in AMP.

She writes on https://amp.dev/documentation/guides-and-tutorials/optimize-and-measure/optimize_amp

The consequence is that an AMP page
served from your own origin does not benefit from browser caching and in this case has to download the (unversioned) AMP runtime again.

If that’s okay, will the browser cache slow down my amp pages?

That is talking about the cache max-age for scripts served from the AMP CDN. This won’t apply to WP Fastest Cache which would only have control over scripts served from your origin. But again, AMP does not allow scripts from your origin (except with for amp-script), so this is irrelevant. Enabling the plugin’s browser cache functionality should not negatively impact AMP pages in any way.