Update: After a month, I still haven’t found a solution to upgrade my blog from WordPress 2.1.3 to 2.3 successfully, while this upgrade is still urgently needed.
@mcbo -> I just upgraded from 2.0.3 -> 2.5.1 without any problems.
Just make sure that you don’t drop the tables as mentioned above, but just empty them…
TRUNCATE wp_terms;
TRUNCATE wp_term_relationships;
TRUNCATE wp_term_taxonomy;
After that just call upgrade.php and it should be okay!
Worked for me just fine… Good luck!