Showing Category & Titles of Posts

I would like to show title of each category, have the ablity to apply styles to it, and then show the title of each post found under that category and make each of the titles a link to the correct post. Anybody know how to do this with either the default template tags or a plugin.

Thanks