Only the fire/mouse buttons go through the 8520/CIA chips. The directions are controlled via Denise and a couple other chips on the motherboard... There is a fuse on the motherboard for the joystick ports. It's at F400, near the back joystick port. It looks like a resistor without any color bands. You can check this with a voltmeter between pins 7 & 8 on the joystick port. (The middle two on the shorter side). There should be 5volts there.
-Jim