I've only seen that with 5 1/4" floppy drives on a pc as the connector wasn't keyed.
Then you are lucky. I recently connected a rather new 2.5" floppy drive and the LED was lit all the time although I was absolutely sure that the cable was connected correctly because I checked that the pin on the connector fitted perfectly into the notch on the drive. When I disassembled it again, I noticed that the drive had a notch
on both sides, so that the cable fits either way.
You still need to be very careful when connecting computer parts, even nowadays as everything seems to be colored or keyed or whatever to be fool-proof.
