Ideally I would want at least 4 (or better 6) physical audio outputs. Which would allow me to route audio in a studio environment... or could be configured as left and right for standard desktop work, or as multichannel surround sound for entertainment.
Each physical output would require it's own channel... you wouldn't want 8 or 16bit support, 24bit is al that is required.
From a hardware point of view, you would probably really want a small "DSP like" processor and a bunch of SigmaDelta DACs... that would be the simplest way.