Is it okay to delete themes directly from folder?

Access your server via SFTP or FTP, or a file manager in your hosting account’s control panel (consult your hosting provider’s documentation for specifics on these), navigate to /wp-content/themes/ and remove the unwanted themes.

Yes, as long as you don’t delete the active theme or the parent of an active child theme.

thank you