Google Map Cripiling site

I am currently using a premium directory theme provided by templatic. This theme includes a google maps widget which shows location of the posts on the homepage.

Mind the locations on the homepage change due to traffic of the website and new additions to the directory but there are never more than 10 map locations to display at a time.

Yet when I upload 1000 directory listings (posts) the google map widget works fine but when I upload over 10,000 the the homepage fails to load unless I removed the google maps widget from the homepage.

I am running a wordpress multisite with buddypress and quick cache plugin.

Does anyone has any idea of how I can fix this?