Hiding previous_image_link() if blank

Hi, I’m using the previous_image_link() on my single attachment page, but because I’ve given the div a background in the CSS, even when there isn’t a previous image to link to, there’s an empty box there.

Is there a way to check if a previous image exists before adding the div, so that it doesn’t show up when the link isn’t there?

Thanks in advance.