I can not reproduce this on my side. I don’t see any red square here, on none of my devices. Therefore I can not really help you with that, sorry.
However – I found the following global CSS rule on your site:
button:hover, html input[type="button"]:hover, input[type="reset"]:hover, input[type="submit"]:hover, a.button:hover, button:focus, html input[type="button"]:focus, input[type="reset"]:focus, input[type="submit"]:focus, a.button:focus, a.wp-block-button__link:not(.has-background):active, a.wp-block-button__link:not(.has-background):focus, a.wp-block-button__link:not(.has-background):hover {
color: var(--base-2);
background-color: rgba(181, 68, 53, 0.85);
}
So the theme or some other plugin overrides the style of hovered buttons (mouse is moving over the button) to a red background. Some themes or plugins use global CSS rules for HTML elements instead of having proper class names, like for <button> in this case. However PhotoSwipe also uses <button> elements for the clickable elements, since this is the correct way to implement that. Having CSS rules with a global scope for elements instead of classes is way too generic and should be avoided. As a workaround you can try the following additional rule for PhotoSwipe:
.pswp__button {
background: rgba(0,0,0,0);
}
Also see your browser development tools ([F12] key) where you can examine all elements and styles on the website. Fixing the global CSS rule to be more specific and not just apply it to all <button> elements would be preferrable.
