This is brand new installation. Straight from the localized, official zip. The English version works just fine.
The issue/bug is that mysql 5.5 and 5.6 does not have UTF8_swedish_ci within UTF8MB4 so therefore this is quite big issue affecting most of nordic/scandinavian sites.