Remove PO info from non PO orders

I’ve noticed that in all orders, at the bottom it shows:

Purchase order information
Purchase order number:
Invoice address:

Is there a way to only show this in orders that actually used the purchase order?

Also, I don’t want the invoice address shown ever. I tried to look for some classes or something to hide it with CSS, but there aren’t any (a div around the whole thing would be useful).