Menu marks more than one category for post as current

My primary nav menu is a list of the categories on the site. So, whenever you are reading a post the category that the post is in is changed to the ‘current’ colours.

My problem is that many posts are in 2 categories, and this results in that the menu marks both of the categories as current and it lights them up as active and current.

How can I make it so that only one of the categories is marked as current the (primary category) in the menu?

I have tried this css, but it won’t work:

.main-navigation .main-nav ul li.current-post-parent ~ .current-post-parent > a {
background-color: transparent;
color: #fff;
}