Custom Terms (Categories)

The list of categories (terms) only seems to be those with an assigned post. Is it possible to list all the category (terms) even if they have no posts attached?

Only that for some reason it caches too well and new categories never seem to appear when a post is assigned to one.