Have you seen typical driver sets for things like graphics cards these days?
Thanks to direct support for CUDA, PhysX, OpenGL1.0 - 3.0, Direct X (up to v10), Pure Video etc, the drivers for my card alone weigh in at around 100 MB, not including tweaking tools.
The linux drivers aren't much smaller.
I don't really think that has anything to do with initalisation though. With drivers your really are getting back to HD speed and CPU speed. Besides even it took a full second to initalize the card, with the Quad core beast, blazing fast bus speeds, and HD speeds you should be able to initialize just about everything at the same time. Aside from the fact that many people, as some have pointed our here, this is possible. Something QNX or Amithlon, take seconds to boot and you can access the internet, hear sound, see the desktop.
For Windows, there's also the speed that "some" apps install. Or some Windows updates. Windows update can take minutes just to figure out what needs to be updated. I can render huge 3D scenes in Maya in less time. What in the world could Windows be doing in all that time?