The WPSSO Core plugin gets data from WordPress and supported 3rd party plugins and services – the add-ons generally only format the data that WPSSO Core has gathered. WPSSO Core should not affect performance at all – it’s one of the fastest (and probably the fastest) meta tag and Schema plugins available.
Are you having performance issues with WordPress and WPSSO? If so, that may be an indication of a badly coded content filter from another plugin or your theme, or a badly configured database. The following troubleshooting guide may help finding the source of the issue: https://wpsso.com/docs/plugins/wpsso/installation/troubleshooting-guide/
js.