As of 3.6, mediaelement.js has been folded into the core of wordpress. I don’t think you’ll experience any nasty side effects if you keep the mediaelement.js plug-in active, but there really doesn’t seem to be any reason to do so.
Note that if you used the alternate shortcode (mejsvideo) rather than the standard [video] shortcode, you’ll have to go in and change things, as the version in wordpress core only uses [video] and [audio] as shortcodes.
Until earlier today I was running off the plug-in using the alternate shortcode. I’ve now switched to the built-in version using the standard shortcode. I had to disable Viper’s Video Quicktags, as its shortcode conflicts with mediaelement’s, but with all the cool new embedding features that are built into wordpress 3.6, I don’t miss it.
Cheers.