the CV64/3D allocates itself 256Mb of space in the Zorro3 address area.
the card itself doesn't have ANY jumpers on it at all. the only pins are the small block of 8/9 at the front of the card (for factory use only) and the philips scenic highway bus (for Mpeg decoders, video digitizers etc.).... oh wait, i think there is one jumper, but its only if the cards Zorro2/3 autosensing didn't work and it just locks the card into Zorro2 mode. so as you have an A4000/030 (which properly supports Zorro3 data burst) you dont need to worry about that, and should get a nice transfer rate to the card.
anyway, if you boot with no startup sequence, then the machine isn't loading the drivers, setting up the screen space, etc etc. in fast ram, so you will probably find yourself with about 4Mb extra.
on a full boot, then yes you will loose about 4Mb of RAM for screen space, and drivers etc. basicly it sounds to me like you need to have a play about and sort out what monitor you need to use out of CGXMode. as at the moment it sounds like you are using someone else's preconfigured monitor definitions, or like you are using a CGX monitor driver that is not right for your monitor.