There was a dedicated adaptor for Amiga called RoboShift and had a
little green LED to confirm it was switched between mouse/joystick.
I unfortunately bought a 2-way adaptor which (when a mouse was plugged
in) would make random inputs that would mess up joystick control.
Go find a RoboShift if you can, they're supposed to be good. Full
buttons etc.
:-)
Possibly even CD32...