If you’ve used global attributes, you’ll be able to change the name under wp-admin > Products > Attributes. However, if you’ve used product level attributes, you’d need to update each product.

If that’s the case, I’d recommend exporting all of the products to CSV and then you can make the change far more quickly. When you’re doing, import the file again, choose to overwrite existing products, and you’ll be all set. You can find more information about the exporter here:
https://docs.woocommerce.com/document/product-csv-importer-exporter/

Cheers!