OK, you have a working A2000 and a GVP SCSI Controller; the GVP is likely set as device 7, what are the HHD & SCSI ID's?
The GVP is self terminated, so for SCSI 2, parity should be set ON for both drives. The most consistent chain is: GVP---HDD---CDROM----TERMINATION (active). If you need a terminator, get one listed as "active," this means it "actively" compensates for signal fluctuations. "On drive termination" often is unreliable. Make sure all connectors on the SCSI cable work; use your working HDD to check that it boots from the different connectors. Google you CDROM's specifications to make sure ther are no other odd jumper settings as "Sectors" is uncommon. The CDROM's drive specs may have other information (and could you tell us the exact model of HDD and CDROM. As above give us the most accurate information on everything; too much information never hurts.
I have an A2000 with GVP controller, HDD, CD/DVD-Rom, and SCSI card reader and all is well. Best guess is Termination issue, bad cable connector, and lastly some "weird Amiga thing."