update:
“Minimum subtotal of matching products” at the bottom of the options is causing the problem. It works when applying but does not remove itself when lowering the number of items below the allowed number or price.
But if I use the “minimum spend” setting at the top of the options then it calculates the cart total rather than the total of that specific product – which is not what I need.