After activating the plugin and seeing the error code, I went to the main page of my website, to see if it was visible on front-end. So I guess that counts as if it didn’t go away after reloading a page.
And yes, it was visible in front-end and in back-end even when I go to another page.