Need for encoding html entities

Thanks for reporting this issue.

I create a sample file with “display_name” = “Mylène” and it was uploaded correctly. Not as anonymous but as Mylène.

Can you please share your file? Please don’t forget to change personal information and anonymize emails before sharing it.

I am very sorry. I have been experimenting a lot and I did not save all file versions that I tried.
I tried to create an example file for you with this problem of encoding and failed. If I leave the characters both in display name and review content the import gives me no problems at the moment.
I am sure I had Anonymous and empty review when I was experimenting and still had the time stamp and the amount of stars correct.
As I need to redo all my imports after the Line Feed problem has been solved (other issue) I will try to identify this problem more exact.
For now you cannot do anything to analyse I am afraid……….

OK, no worries. I will then mark this topic as resolved.

OK, I reloaded all reviews to see that I had several reviews for Anonymous and others with empty review text.
So I made 2 csv files that show both issues.

https://www.dropbox.com/s/izp1afn4n9ganil/review-import-bugtest2.csv?dl=0
results in Anonymous

https://www.dropbox.com/s/hjynvqs11rtbwch/review-import-bugtest3.csv?dl=0
results in empty text

In both cases I see no errors (not in the log either).
So in essence these are 3 problems because you would expect to get some warning or error. Don’t you agree?

I reopened the topic.

I have encoded both files (first one the userid and second one the review text) and then there are no problems after importing.
So there is definitely something wrong with processing of some characters. In this case character é.

https://www.dropbox.com/s/lwzmh2wil12b77d/review-import-bugtest2_encoded.csv?dl=0

https://www.dropbox.com/s/9qp92wb4myc6ltk/review-import-bugtest3_encoded.csv?dl=0

Can you please look into it?

The problem is that your files are in Windows-1252 encoding. The template file has UTF-8 encoding, so the plugin don’t expect users to upload files in different encodings.

UTF-8 supports all special characters and it is the encoding used by WordPress.

Please update the plugin to version 3.56. We added a check of file encoding and conversion of Windows-1252 to UTF-8.

First of all I can report that the problem is solved! So thanks for that.

I started out with an UTF-16 unload of a table and worked it through Libre Office (to also get the comma as delimiter). Why it is saved as Windows-1252 I do not know.
Even so I think a warning would be in order that someone is uploading a file in the wrong encoding. Something like are you sure you want to process this file?

And again I would like to advocate the option to write something in the log (maybe with some Test mode option?) so web site builders can see what is going on.

All in all I will add these remarks to the review (also the fact that there is no option of specifying what delimiter the user has used when trying to import a file).

But thanks again for solving my problem, I now can go forward with the migration to Woocommerce!

Great, thanks for the confirmation!

Unfortunately, I’m not an expert in Libre Office software and cannot answer your question about choosing an encoding there.

The only supported way of creating files for upload of reviews is by downloading and modifying the template supplied with the plugin. This way, you will have the file with the correct encoding that is supported by the plugin. It looks like you didn’t follow this process and created your own file. Hence, it didn’t work.

Unfortunately, it is not possible to reliably determine encoding and generate a warning programmatically. Even advanced software packages like MS Word often get text encodings wrong and ask users to manually choose a correct encoding.

I didn’t fully understand your comment about log. The plugin already displays a log after upload.

Can you please explain why would you need an option for choosing a delimiter in the plugin? It looks like you are trying to hack the plugin again and use it in a way that it was not designed for. Please use the template for the upload. It already has the correct delimiter.

I hope this helps ??

I am using your plug-in to migrate reviews from another platform (in this case OpenCart) to WooCommerce. For that I need to create my own version of the csv by using Excel and some functions to finally get to the lay-out your plug-in accepts to import. This is not hacking or whatsoever just using the import function.
Problem is that European Excel ONLY uses ; as a delimiter and you cannot change that (there are a lot of solutions to get around this on the net but also reports that these solutions are no longer working). If you use LibreOffice you can set that delimiter to comma or whatever you want.
Suppose someone wants to edit the exported reviews using Excel? When reading the file you can tell Excel to use the comma as the input delimiter but when you save the file to csv you cannot any longer. So problems for us European users….. ??
That is the sole reason I would like to be able to set my delimiter.
Does this description make my request more clear?

Then about the log. You only report some errors (like review already exists) after the export on a WordPress screen. In my view this is not the log. I mean the debug log set in WordPress. I can imagine that you do not want to write to that log always so combining it with a Test mode to report any warnings or errors would be a great addition.
Suppose you are importing a file with wrong encoding and processing the user leads to Anonymous. Surely you want to report this in some way? How would the user know if the import contains a lot of reviews?
Hope this makes it more clear too.

• This reply was modified 6 years, 1 month ago by Harm10.

Thanks for the additional information.

Is there no way to open and edit CSV files with commas in European Excel? It is such a powerful application.

If we are to add support for semicolons as separators, it will be necessary to provide the correct template (with commas or semicolons) in the plugin. Unfortunately, the plugin cannot know what kind of Excel a user has and which template to offer.

As I mentioned earlier, it is not possible to reliably determine encoding of a file and generate a warning programmatically. So, we won’t know that name of the customer couldn’t be read correctly. I hope this makes sense.

As I said just look on the net to see what people think of Excel on this topic…… ??
But people use LibreOffice! It is free and as of now I haven’t seen any Excel function that it does not support.

I do not agree that you cannot know which separator was used. If you make this a configurable option it is the user’s responsibility to select the right value. If the value is wrong than reading the header will fail directly.

And I still do not get that you do not understand my wish to be able to do some logging. WordPress has a general logging option and when switched on you can easily write messages to it (I use that feature myself and several other plug-ins do as well). I know plain customers do not want to look into these messages but site developers do I can assure you.

I would also like to repeat that I really appreciate your support. A lot of plug-in developers can take an example on that!