Actually, this seems to only happen with the side inputs on my tv. The back ones are fine.
BTW, I was plugging my amiga into the tv's rear inputs, and I heard a buzzing noise from the tv as I accidently tried to plug the audio prong into the video jack. The amiga was off (power supply was plugged in but switched off), and I had left the tv on. Wikipedia says that RCA cables can cause equipment damage because the hot connection makes contact before the cold screening connection. However, I would think because the amiga was turned off it wouldn't hurt anything right?