filters based on categories

I am in for this either!

The WooCommerce Ajax Layered Navigation by WooThemes works like this.

It won’t show a filter in a product category (or any archive view) unless the products in it have the relevant attribute.

In your case you would add filters for all 4 – size, material, color, gender – and if shoes don’t have a material or gender attribute the filters won’t be displayed for those two attributes.