Once you've converted to RGB, the video takes much more memory and bandwidth.
Yuv (or ycbcr) is skipping many color components.
RGB is 4:4:4 (for every 4 pixels, there are 4 sames of each component)
Yuv can be 4:2:2, 4:1:1, 4:2:0 (really a misnomer), etc. so it's much smaller in memory.
DVD, BluRay and all the common mp4/h264 variants are 4:2:0 for example.
Not having to convert to a larger more cumbersome RGB format is one of the biggest benefits of overlay, 3D "faked" or not.
