This is a fair point. The only thing I'd say is, why can the commentator's and whoever else hear the audio but the viewer can't?
I mean, we're allowed to hear it in Cricket and we can hear it in Rugby. When we do, we can make more sense of the decisions because we can hear them explain how they perceive things.
Rugby has it absolutely spot on in that we, the viewer, can hear the referee all the time too. That's something our sport has alwaya lacked.
I feel like the relationship between fans and officials would build more bridges if we were privy to hearing what was said at all times, not just VAR audio, but the lot.