Is it possible to customize message in cart

You can do it on the plugin settings at WooCommerce > Settings > Products > Product Assembly / Gift Wrap / … > Message: “{checkbox} Assembly: Yes ({price} / unit)”

No, I want to change the message “Assembly: Yes (xxx / unit)” on the cart page to something else

Update to version 3.1 and use the product_assembly_cart_data filter.
Here’s an example: https://gist.github.com/webdados/1b14574564e9ef0d853a5c1d43a9ed3c