You have been reporting a loud hard drive and strange floppy behavior all along. Unless you are running a strange trap-door memory card, like the one from ICD, there should be no memory conflict with fast memory.
Replacing the hard drive would solve one problem. Mating the GVP to the A500+ might solve the other if its floppy is more reliable. Yes, you can still load Workbench 1.3 to the HDD while running 2.x roms (for compatibility with both roms).
As you may notice from other threads here, replacement floppy drives are in short supply. Maybe a cleaner disk will help.
This would be a no-brainer for me. The A500+ with greater memory and newer rom would make better use of the GVP