Conditional Block Asset enqueue in block.json

Css file included in block.json is getting enqueued in frontend even though block is not being used. This is the case in classic wordpress theme but works as expected in block theme (example: in twenty twenty 4 theme).

Any workaround optimization for this in classic theme?