or would the game switch to the RGB port bypassing the graphics card?
It would.
But then, there's always some tricks to route the picture back to graphics card display, if you don't want to use separate monitor for that. You might need compromise about picture quality, but anyway.
In AmigaFun's Mediator setup, you can for example have TV-card plugged in and feed the Amiga's native modes to it and then you can display those games on graphic card's display. Mediator keeps displaying the TV even when games disable the multitasking.
Or if you get Amiga's graphic card, which has scandoubler and use it on Amiga which has videoslot. Then you can have all native modes displayed on same monitor with gfx card. Or graphics card with passthrough and external scandoubler. Or some monitor switcher system.. or dual input on your monitor, which you're probably already doing as you have external scandoubler too
