Update:
I have noticed that my server is using php-fpm 7.4 by default (both 7.3 and 7.4 processes started and running), but my nginx config is set to 7.3. I have updated the nginx config to use 7.4, then checked for the issue – and now I cannot reproduce it. The _wp_http_referer parameter is also absent from the url request.
So problem solved, by accident