I'm sorry but the only way you can get those screenmodes to work without a scandoubler from an A1200 is with a good multisync monitor, and both of them (800x600 and 1024x768) will flicker like mad without a flicker fixer. I'm afraid to get Workbench in those resolutions with a regular TFT monitor you will need the Indivision. Perhaps people will be selling them cheap once the Indivision Mk2 comes out with its Graffiti emulation and DVI output.
The adapter you use should be fine for running Workbench in 640x512 in PAL, maybe it will even fit a little bit more with overscan. I use a RGB-S-Video adapter plugged into my LCD TV and run Workbench and lots of programs in this resolution, it's not too bad if you put your programs on different screens.
Since you have that Xbox, have you set up UAE or AmigaSYS with it? You might be able to use HighGFX mode through that, and it'd make a pretty decent emulated Amiga system. Try it out if you haven't already -
http://amigasys.ultimateamiga.co.uk/xbox.html