I am thinking your PSU cannot supply all of the power it needs for your whole Amiga. I know you have to have like a Bigfoot one like I have.
Buying a bigger PSU is better. And for your floppy drive, have it connected to the new PSU if you get one. I think a case solution would solve all of your problems.
If someone thinks its someething else, please correct me.