My XE G4 has been out of action for years now, but back when it worked, I did use a Radeon 9250 in the AGP slot with no obvious problems. My machine had not been "fixed", it was in the original state.
And I never heard that it should be a problem to use the AGP slot in that situation - but that doesn't necessarily mean it isn't :-).
IIRC, I used an SiI0680 and an SiI3114, and also the internal ports for CD and a boot HD (the internal ports of course in PIO mode).
Best regards,
Niels