@bpebble – You may want to look at how many of the existing CSS frameworks create their responsive modals. We use the same methods, and in fact took them initially from Twitter Bootstrap.
Also your under the assumption that to be responsive it must use EM based unit measurements. This is false, in fact responsive in general simply means that the design is built to render differently based on screen or device size. There are no rules as to how this should be implemented. Some frameworks do it with EMs, some with % and others simply create multiple media queries and use px based sizes that are predefined.
The last using PX is not fit with todays methods, but the core concepts stated in [5] specifically state “A flexible grid-based layout that uses relative sizing”. Percentages are just that, relative to their parent.
I would be happy to help you get it working the way you want, as the plugin is already set up to be customized in any way needed, but you are free to use another plugin if that is your wish.