A search on google revealed an ad on computercity:
Competition Pro CD32 Joypad, supports all 6 buttons. Extra buttons also work on A1200 and A4000. With auto-fire functions.
look at
www.computercity.bizApearently there are 6 buttons, and the same pad will also work on other amigas, so there is no HW difference in the CD32 joystick port for the extra buttons.
But it seems like the easiest solution is to buy a joypad..