It it might be the caption background that’s causing the grey border on the white background. If you’re not already using one, or working with a child theme, you might want to try installing a custom CSS plugin so any future updates won’t disturb your changes. Examples: https://www.ads-software.com/plugins/search/css/
I think the Twenty Eleven theme uses .wp-caption { background: #eee;
which causes a grey border. Once you enable your CSS plugin, try using .wp-caption { background: #fff;
instead. I think that might change the border color to white.
]]>