Bandwidth is the key, if you keep it below 1500kbits, you are in safe area. The Vampire demos are all around 640x360x1500kbits.
And they play fine, there was even a x16 core for the V4 tested. If you want to decode the stream, you most certainly don't want to do this yourself, but instead use a better gfx card with FPGA/ARM, like the ZZ9000, and let it do the work for you.