Related products nonsense

Hello,
I have a weird problem with how woocommerce shows related products on product pages, all of the products have the same first category but their primary category is set to the lowest one in hierarchy (most specific), so I would expect to get related products based on the primary category.

But instead always the same few (low cost) products are repeating in this list even on products with 10-20 times the price . Is there a way to fix this?

(Example: I have a suit/dress product and a tshirt, both of them share first category (clothes) but have respective categories. And when I view a suit product I see tshirts in related products)