Category Pagination / Custom Permalink

I am using this plugin, but using a Custom Permalink for my site

I have it so that “category” is not listed in the URL.

I have it set up so that:

domain/categoryname/ is the permalink
but
domain/category/categoryname/ still works properly

If i goto
domain/category/categoryname/ < — wp-navi works correctly and can will goto
domain/category/categoryname/page/2/ without issues

however:
domain/categoryname/ <— wp-navi breaks (404) when I click on the next page/arrow to goto
domain/categoryname/page/2/

How can I fix it so that “category” isn’t in the URL?

https://www.ads-software.com/plugins/wp-pagenavi/