Instead of Exerpts or posts can I display just post titles?

I’ve spent the better part of the last hour and a half poking in the codex and I just can’t seem to figure out how to do what I want. In my Category archives I just want to list ALL post titles for that category. Not the exerpt, not the post, *just* the Post Title linked to the permalink. Can it be done without that admin setting to display only X number of whatever per page screwing it up? And what template tag do I need to look at for this?