I think the duplicate image/ thumbnail is being created when the page is refreshed.
The instinct of most users is to refresh the page when they don’t see their thumbnail appear. But the plugin does not seem to clear the last upload out of the browser cache because when you refresh the page, you get the “Are you sure you want to send this form again?” message.
…but when you go ahead and refresh, it re-uploads the image and creates the duplicate thumbnail.
This will continue happening unless you leave the page and come back to the gallery page.