Remove WP My Account for Logged Out Users

I’m customizing my admin bar and one problem I have come across is the fact there is a default gravatar at the top of the screen. (see screenshot)

I know the gravatar is connected to my accounts. I can’t remove my accounts because I need it for users that are logged in. I would like to only remove it for logged out users. I’ve tried using the code:

function admin_bar_logout_avatar( $wp_admin_bar ) {

if (is_user_logged_in())
        return false; {

$wp_admin_bar->remove_node('my-account');
		}
		}
add_action( 'admin_bar_menu', 'admin_bar_logout_avatar');

But that doesn’t work. How can I change the code above to make it work so that users that are not logged in do not see the my account gravatar?