@Patrick:
I have similar problem: RGB output is connected to the CV64 input with a RGB-VGA cable. I've recently upgraded my A4000D mobo to 3.1 ROMs and after I powered it up, it crashed in a loop.
Similar like you, after I disconnected the cable from RGB port, Amiga started to work without problems.
What's more strange, after installing the old 3.0 ROMs again, the problem disappeared, but if I try to use BlizKick ans a 3.1 ROMs image file, the problem comes back.
Any ideas?