[Next GEN Gallery Plugin] resize images, jagged lines! HELP~

I am having problems with resizing images. I usually use a highslide effect for photo view. Most of my horizontal photos turns out to be fine, but when a photo is vertical, my photos starts to get jagged lines and oversharpered. I am not sure what is the correct terminology to use here, but its like converting my images into smaller size and ruin my photographs.

How should I resize my images before uploading? Or do I have to change some settings?

Here is my website:
https://www.edward-wong.com

Thank you very much for your help!!!