Weird database character encoding

  • Using Duplicator v0.5.12 and loving it. I am migrating a number of sites and I noticed that some characters in the posts are not being encoded properly.
    For example I get:

    a€? in place of ‘
    a€“ in place of –
    ? in place of space (beginning or ending)
    a€? in place of opening “
    a€ in place of closing “
    So the sentence
    “If the park closes,” said Simitian, “that’s 400 low-income folks who are out on the street.
    comes out as
    ? a€?If the park closes,a€? said Simitian, a€?thata€?s 400 low-income folks who are out on the street.

    The installer tells me the encoding is utf8 and the collation is utf8_general_ci. This agrees with the MySQL database. What’s going on?

    https://www.ads-software.com/plugins/duplicator/

Viewing 9 replies - 1 through 9 (of 9 total)

  • Hey mauryg,

    Checkout this question on the FAQs page I believe it may help or provide some clues about this issue:
    -> Browse to: https://lifeinthegrid.com/duplicator-faq
    -> Find question: “How can I fix international characters issues?”

    Right now its difficult to spot these issues. Sometimes the database.sql file that is written to the archive.zip file can actually be written as ASCII, based on how PHP is configured. This can mess things up as well, I’m not saying that is the issue in your case but it has been for others in the past. I have not found an issue around that exact problem quite yet…

    Cheers~

    Thread Starter mauryg

    (@mauryg)

    Hi Cory,
    Thanks for the fast reply. Actually when I looked at the FAQs it appeared that the item on “How can I fix strange characters” was more relevant. All of my articles are in English only.
    I examined my database with phpmyadmin and the encoding is utf8 and the collation is utf8_geneeral_ci. I was tempted to add the correct collation to the wp-config file and see if that made a difference. It’s also interesting that while most of the 30 individual MyISAM tables are utf8/utf8_general_ci, 4 of them, associated with particular plugins, are listed as latin1_swedish_ci. And the SUM is latin1_swedish_ci. But those plugins are not involved with the posts in question.
    I tried changing the collation parameter when I did the install but that didn’t help.
    The problem was random and did not occur in all posts/pages. I wonder if the affected text may have been copied from MS Word or similar source and the internal formatting was the culprit. Worth a try. I’ll run a test install and let you know.

    Maury

    Hi Mauryg,

    Sound like your heading in the right direction. Character encoding issues are very difficult to pin-point. I have spent hours trying to get things going on several system and it can take allot of time troubleshooting, this is why I normally just out-source these types of tasks…

    Hope you make some headway…

    Thread Starter mauryg

    (@mauryg)

    Hi Cory,
    Haven’t had a chance to test all my ideas but just a quick observation. I took the one website that gave me the most problems and used WP-Migrate-DB to export a copy of the database only, then imported it into the new site I had migrated with Duplicator. Weird characters were gone. So clearly something was screwed up in the encoding.

    Maury

    Yeah I do plans to try and dig further into this at some point, just need more time…

    Thanks Cory, I’m doing without RSS feeds for now but it would be nice to figure this one out. I restored a previous database from before the characters showed up but it’s still doing it. Ugh.

    Ooops I just noticed this wasn’t my post ha ha ha, same issue though or a cousin of it anyway….

    I think the best way for me to help out with these issues is submit a help ticket. This way I can work with your setup and try to pin point what issues are causing the encoding issue…

    How do I put a trouble ticket in? I’ve been looking for a way to do that but have had no luck.

    Just click on the link above and then click the link “submit a ticket” are you unable to see the website?

Viewing 9 replies - 1 through 9 (of 9 total)
  • The topic ‘Weird database character encoding’ is closed to new replies.