Apostrophe Issues

When I publish a post that contains apostrophe wordpress is adding an additional apostrophe.

For example: If I write

Joe’s Crab Shack

it becomes

Joe”s Crab Shack

when I view the blog post.

If I edit the post, to fix this problem it adds yet another apostrophe. In fact they keep adding on anytime I edit posts with apostrophes.

Others are having this issue also see https://www.ads-software.com/support/topic/44574 however there doesn’t seem to be a solution.

I’ve started a new blog from scratch with no plugin activated and the problem still exists. I tried 2.0.5 with no success then tried 2.1 alpha 3 just for the hell of it and same issue happens. This leads me to belive that my web host has set an option on Mysql or PHP.

Anyone with light to shed on this?

Thanks.