Disconnecting a joystick while the miggie is on is perfectly safe as long as you don't have any of the fire buttons or directional switches pressed on when doing so... 
That wouldn't do any harm either. The joystick is either connected or it's not. In either case, pressing the buttons and moving the joystick won't cause any harm. The 5V used in the port isn't enough to arc or cause back EMF, so the actual process of disconnection doesn't present any special danger.
