What model of "one for all" tv antenna is it?
If the 'booster' is a module that came with it, it might actually be a power injector for some electronics in the antenna head, typically a small amplifier/filter. In which case the aerial won't work as intended without it. Losing sound could be an artefact of too much signal!!
But of course if it's just a regular signal booster, ignore me!