CD32 pads are read serially. You have to specifically reprogram some of the gameport lines to allow serially clocking the data (off the top of my head, the primary button line is turned into an output used for the serial clock, while one of the pot lines is used to read the serial data in fast mode). It sounds like the game isn't reprogramming the gameport lines correctly. Running another game first sets the register that payback doesn't. At least, that's what it sounds like to me.
