Just found a solution if someone finds this post later down the line.
Apparently it’s not something that changed in WordPress but rather in how I packaged the plugin.
When packaging the plugin make sure inside the zip archive you have a folder, not just the plugin files. It looks like when there are files inside the archive WordPress leaves the temporary folder as is, but if you have only one folder inside (which contains the plugin files) then WordPress will move that folder up one level to the wp-content/plugins directory and remove the temporary folder.
