Directory loads slower when logged out.

Something is puzzling me. I’ve noticed that when I’m logged in to admin, member directories load faster than when I’m logged out. Almost 50% of the loading time. I’ve turned off all other plugins and tried on a different theme as well. No caching. Same result.

So, 2 questions:-

1) Is anyone else experiencing the same?
2) Is there any explanation for this?

Thanks