If your GVP accelerator is one of those "Combo" units, move the SCSI chain to it and extract the HD+8 out. Then try to source some more GVP SIMM to expand the memory.
I know GVP SIMM are rare than hen's teeth, but if you want those desperately, try the GVP SIMM hack on aminet ("transform" a regular 60ns 72-pin SIMM into a GVP one).