Add sidebar to a category page

We have a blog on our website with a sidebar. On the sidebar, I’ve added the categories widget so that people can filter the posts. However, when you click on a category, the sidebar disappears and you only get the list of blog posts within that category. How can I make the sidebar stay? This is not a page in my pages menu so I don’t know how to edit it.

Thanks in advance.