Color Picker limited to admin user

I have changed the theme.json so that custom colors are only accessible to admins. Unfortunately, the GenerateBlocks blocks don’t seem to use the default WordPress color picker? It works for all other blocks. How can I limit the GenerateBlocks color picker too?

function bw_change_theme_json_settings( WP_Theme_JSON_Data $theme_json ): WP_Theme_JSON_Data {

	// New settings
	$new_data = array(
		'version'  => 3,
		'settings' => array(
			'color' => array(
				'custom'         => current_user_can( 'manage_options' ),
				'customDuotone'  => current_user_can( 'manage_options' ),
				'customGradient' => current_user_can( 'manage_options' ),
			),
		),
	);

	// Merge and return settings
	return $theme_json->update_with( $new_data );
}

add_filter( 'wp_theme_json_data_user', 'bw_change_theme_json_settings', 100 );

Thanks,
Manja