That's right - there are two perils when 'hot' plugging devices:
- shorting powered pins to ground (causing a fuse or track to burn) or an I/O pin that can't cope with it (destroying some active component)
- static electricity: even when not noticable to humans, static buildup (of a non-earthed device or body) of a few hundred to several thousand!(!) volts can instantly destroy electronics
While the Amiga's joystick/mouse ports are explicitly hardened against the latter danger (see manual) there still is some danger, so handle with care!