From what I have read I think the GVP RAM-Modules have some contactproblems, or one of then is broken. See if the GVP-RAM sit tightly in their sockets. Maybe remove the RAM and clean the contact with pure alcohol useing a Q-tip.
Try running mit the GVP install but without RAM on the card (the first bank is soldered directly to the board I belive, of course you can`t remove that).
Maybe the RAM-modules are too slow, try deactivateing the burstmode, or nowaitstates with the jumpers. I will cost you a little bit of speed, but your system should be more stable after that.
Regarding RAM on ZII-Cards: The ZII Adressspace can only adress 8 MB, so the total of all the ram on all ZII cards should not exceede 8 MB. The GVP (and most of the other accelerators) put their memmory outside of the ZII autoconfig adresspace, so there is no problem.