Here's the A500 PDF Schematic (page 6 is what you're after):
http://www.amigawiki.de/dnl/schematics/A500_R6.pdfIt could be one of the following components:
- Bad/Damaged U15 (74LS157 Quad 2to1 MUX)
- Dry solder joint on one of the RP401 pins (4.7K pull-up resistor pack)
- Bad/Damaged Denise chip (more likely a bad connection)
I don't think it'd be one of the EMI components, most likely you have a dry solder joint between CN1 and Denise, or U15 has become faulty
Edit: I guess it could also be a fault with U33 (74F04 Hex Inverter) or E109 (47pF cap) - This line controls the select input of U15 - The failure of U33 would cause errors sampling left and right joy ports and make the mouse move intermittently
