I get what it is, I’m not sure how it got that way if it is not defined on the permalinks setting screen. The best approach is to reverse whatever introduced the permastruct to begin with by using a later hook priority. If add_permastruct() was called from init action, priority 10, then call it again with the desired permastruct from init action with priority 20.
Another approach is to fiddle with the global $wp_rewrite object’s properties directly. This still needs to be done with a later priority to the same hook used to add the taxonomy term element to start with. If it’s not clear what was done, the original call was likely “init” action priority 10. Visit the permalinks settings screen after making changes to update the rewrite rules.