Surely all the RF modulator is doing is giving you an RF signal with video/audio combined, why not just use the Amiga's RF out?
Failing that, connect the Amiga's RF out to a TV and take your audio from the TV's headphone socket. At this stage you can Y it to mono if it's not already.
Cheers