I’m experiencing the same admin-side slow-down accompanied by 502 & 504 errors for some requests; sometimes the entire page fails to load, sometimes only certain pieces.
The issue has persisted across WordPress v3.8.3 & 3.9, and WordPress SEO v1.5.2.7 and v1.5.2.8. I’m having difficulty reproducing it on a duplicate instance. The server environment across the two instances is identical, but in one instance there is no issue, and in the other intermittent partial to full failure.
The requests most likely to fail are load-scripts.php and load-styles.php for a given admin page.