The above solution will work, but you can also use the opacity feature. Opacity will leave the button there, but nobody will be able to see it (so it will keep the spacing). You can also use CSS in your child theme to control the padding around the links, so that the height of the boxes on the right are equal to your image height.
To make the button go away you can do what rossmarc described above.
To make it invisibible, but keep the spacing you can try this:
.right-front-page-boxes a span img{opacity: 0;}
To change the padding around the top link you can try this:
.right-front-page-boxes #top a{padding:10px; 25px;}
And for the bottom boxes you can try this:
.right-front-page-boxes a{padding:8px; 25px;}
You’ll just need to change the padding height from 10px and 8px to something else.