I have the same problem, but with German localization: By default a WordPress installation with DE localization replaces ” quotes with ? (at quote start) and “ (at quote end). If Wp Statistics is active, this is broken and standard English ” quotes appear.
It seems your plugin is interfering with the wp localization function that does this replacement (whatever it might be…). I tried switching off showing WP statistics reports in the post area, but it had no effect unfortunately. Really bad, I like WP Statistics very much, but since this bug directly affects the front end and looks unprofessional to site visitors, I cannot use it unforutunately.