It won’t work if it is the first on the line because some of the definitions in vars.php are prefixed with a space. You can alter this behaviour in your wp-config file — just add an array of smileys and their associated icons, without the spaces in front of the emoticons.
Somewhere above the line that says ‘Stop editing,’ put the following:
// smiley array
$wpsmiliestrans = array (
":D" => "grinny.gif",
":-D" => "grinny.gif",
":grin:" => "grinny.gif",
":)" => "smiley.gif",
":-)" => "smiley.gif",
":smile:" => "smiley.gif",
// (keep adding smileys to suit)
":mrgreen:" => "icon_mrgreen.gif",
);
Note the lack of spaces in front.
You can copy the array from vars.php and remove those spaces if you need to remember which phrase triggers each icon. It’s easier than editing vars.php, cause that will get overwritten next time you apply an upgrade.
(ETA: this at least worked fine for me; if it buggers things for you, I apologise.)