In case someone comes accross this thread, here’s a workaround that I am using.
I installed the Faceted Search plugin and edited the button text and look of the plugin.
It’s now a checklist of my categories that the user can select/deselect and then press “update” (instead of “search”), which will show all posts in those categories (I’ve removed the search field).
The plugin keeps the boxes checked, so the user can see what categories they are browsing through.