Okay, this issue is solved. I created a child theme and copied header.php and footer.php to the child theme folder on the server.
From within WordPress I basically deleted almost everything from those 2 files between the respective header and footer tags. That got rid of everything, maybe also stuff that I might need but that’s something I’ll find out later.
Thanks for the help!