I know of two places where you need to apply such enforcement. One is the password reset page, which also covers user registrations. For this, use the ‘validate_password_reset’ action. If the password does not comply, use the passed WP_Error object to add an error type and error message using the WP_Error::add() method. Adding an error reloads the reset form with any added error messages displayed.
The other is the profile/user edit admin screen. For this, the ‘check_passwords’ action is available, but it doesn’t have any mechanism to add errors. All you could do is wp_die() with your message. I suggest you use the ‘user_profile_update_errors’ filter instead. With this filter, the WP_Error object is passed and you can add an error as is done with the reset form. The difference here is the password is passed as part of the a user object. This is not a WP_User instance, rather a PHP standard class object. The password submitted is $user->user_pass
.