Making use of a custom shortcode in dialog

That’s possible, but I’m not sure why you want to direct a user that way. The post restricted message displays in place of blocked content along with a login form as long as you’ve left that enabled in the plugin’s settings. If they log in there, they’ll end up on the post without the need for redirection.

Or alternatively, if you redirect blocked content to the defined login page in the settings using wpmem_redirect_to_login() it will redirect the user back to the original page. There’s an example in the documentation for this function: https://rocketgeek.com/plugins/wp-members/docs/api-functions/wpmem_redirect_to_login/

If you must add a shortcode to it, you can filter the post restricted message with wpmem_restricted_msg – https://rocketgeek.com/plugins/wp-members/docs/filter-hooks/wpmem_restricted_msg/