Old Version of Image Still Showing

When I delete an image and reupload a new version of it, WordPress still shows the old version as long as the name of the new one is the same. I did flush the cache and clear the browser cache to no avail. It’s a bit annoying to have to rename the image when reuploading it. Thanks for any suggestions!