Ah okay, I was under the impression that maybe you wanted to manually get all the links and just make your own CSV. Like if you used Microsoft Excel or Mac’s Numbers, put all your data in an excel sheet, and then just export to CSV.
I’d aim to search for something will specifically export attachment posts types since all attachments in the media library are a post type. Or something that says it will export media without the content. I’ve been searching but I can’t seem to find a reliable plugin for this, hence why I suggested just creating your own CSV.
If you just need the urls and nothing else, it might be a bit heavy to install a CSV plugin that will export entire post content just for media urls. It’s tedious, but you may be better off manually creating your CSV and using the Admin Columns plugin just to make grabbing those urls for copy/paste easier.