I agree completely, though I'd add that it's also unpleasant. There's a reason some browser and browser extensions try to help track down which tabs are producing audio.

Perhaps GIFV or equivalent should be defined as either requiring the browser not to play audio or as making audio optional, but required to default to off at start of play. The latter would require a minimal transport control, but I could live with a transparent mouseover-only volume button. There's no technical reason that a GIF successor must include or play audio. For that matter it would make a greasemonkey script or perhaps extension that forced sound off for regular mp4 much easier to write in this case.