[Plugin: CSV 2 POST] Problem by importing special caracters from csv

Right now there are no functions to handle these characters. I do have the functions but so far no users have required any type of character encoding etc.

I will add it to the todo list. It is not something that can be added quickly as I need to run all content through the functions.

Hmmm. Don′t you use UTF8 encoding. It′s better for worldwide applications…

Sure.

Here is the functions. They worked fine in my other plugin but I’ll still need to go over them line by line…

// Character Encoding Input
$eci_o42chars['in'] = array(
    chr(196), chr(228), chr(214), chr(246), chr(220), chr(252), chr(223)
);
$eci_o42chars['ecto'] = array(
    '???', '???', '???', '???', '???', '???', '???'
);
$eci_o42chars['html'] = array(
    'Ä', 'ä', 'ö', 'ö', 'Ü', 'ü', 'ß'
);
$eci_o42chars['utf8'] = array(
    utf8_encode('???'), utf8_encode('???'), utf8_encode('???'), utf8_encode('???'),
    utf8_encode('???'), utf8_encode('???'), utf8_encode('???')
);
$eci_o42chars['perma'] = array(
    'Ae', 'ae', 'Oe', 'oe', 'Ue', 'ue', 'ss'
);

// Character Encoding Ouput
$eci_o42chars['post'] = array(
    '???', '???', '???', '???', 'Uuml;', '???', '???'
);
$eci_o42chars['feed'] = array(
    'Ä', 'ä', 'Ö', 'ö', 'Ü', 'ü', 'ß'
);

# Applys Encoding To Entire $my_post Object Based On Settings
function eci_postencoding( $csv,$my_post,$set ){
	// title encoding
	if( isset( $set['titleencoding'] ) && $set['titleencoding'] != 'None' )
	{
		if( $set['titleencoding'] == 'UTF8Standard' )
		{
			$my_post['title'] = utf8_encode( $my_post['title'] );
		}
		elseif( $set['titleencoding'] == 'UTF8Full' )
		{
			$my_post['title'] = eci_encodingclean_content( $my_post['title'] );
		}
	}

	// post content encoding
	if( isset( $set['contentencoding'] ) && $set['contentencoding'] != 'None' )
	{
		if( $set['contentencoding'] == 'UTF8Standard' )
		{
			$my_post['title'] = utf8_encode( $my_post['title'] );
		}
		elseif( $set['contentencoding'] == 'UTF8Full' )
		{
			$my_post['title'] = eci_encodingclean_content( $my_post['title'] );
		}
	}

	// permalink encoding
	if( isset( $set['permalinkencoding'] ) && $set['permalinkencoding'] != 'None' )
	{
		if( $set['permalinkencoding'] == 'UTF8Standard' )
		{
			$my_post['title'] = utf8_encode( $my_post['title'] );
		}
		elseif( $set['permalinkencoding'] == 'UTF8Full' )
		{
			$my_post['title'] = eci_encodingclean_permalinks( $my_post['title'] );
		}
	}

	return $my_post;
}

# Converts Special Characters Using Correct Encoding Values For Permalinks
function eci_encodingclean_permalinks($title){
    global $eci_o42chars;

    if ( seems_utf8($title) )
	{
		$invalid_latin_chars = array(chr(197).chr(146) => 'OE', chr(197).chr(147) => 'oe', chr(197).chr(160) => 'S', chr(197).chr(189) => 'Z', chr(197).chr(161) => 's', chr(197).chr(190) => 'z', chr(226).chr(130).chr(172) => 'E');
		$title = utf8_decode(strtr($title, $invalid_latin_chars));
    }

    $title = str_replace($eci_o42chars['ecto'], $eci_o42chars['perma'], $title);
    $title = str_replace($eci_o42chars['in'], $eci_o42chars['perma'], $title);
    $title = str_replace($eci_o42chars['html'], $eci_o42chars['perma'], $title);
    $title = sanitize_title_with_dashes($title);
    return $title;
}

# Converts Special Characters Using Correct Encoding Values For Content
function eci_encodingclean_content( $content) {
    global $eci_o42chars;

    if ( strtoupper( get_option('blog_charset' )) == 'UTF-8')
	{
 		$content = str_replace($eci_o42chars['utf8'], $eci_o42chars['feed'], $content);
    }

    $content = str_replace($eci_o42chars['ecto'], $eci_o42chars['feed'], $content);
    $content = str_replace($eci_o42chars['in'], $eci_o42chars['feed'], $content);

    return $content;
}

Blame on me: I don′t use UTF-8 for my .csv file.

Excel exports by default to ANSI coding.

Now I re-encode the .csv to UTF-8 and have no more problem with my special characters.

maybe you can place a hint near the .csv Upload button to use UTF-8 encoding (and not native Excel export function).

Best regards,
kai

Hi Kai,

I am using now utf-8 but when importing it drops everyline but in ansi it works fine (but the special characters).

Which delimiters do you use??? double quote and comma? ” and , ??

Hello Antorome,

I exported my data from an excel file using “Save as CSV (MS DOS)”.

This one is ANSI.

After I read your hint I downloaded a freeware text editor who can convert to UTF-8 . It was PSPad ( https://www.pspad.com/ )

After loading my excel generated csv I convert to UTF-8 and save it again.

Import works fine.

The delimiters are the excel default: ; and (maybe) ‘ – but quote was not used…

Thanks for the answer, I have been trying these days to solve it but noway…

Hmmm… did your DB runs UTF8 ?