Wouldn't the rest of the board still be operating at it's normal slower speed?
That depends on how you decide to overclock, don't it? :-)
Overclocking the FSB is generally more tricky than CPU overclocking (unless you have to do some hardware hacking on the CPU itself, then that's harder :-)), and usually you can't overclock the FSB that much, as you still have to maintain timings with other devices in use, so the benefit generally isn't that great.
I never noticed a difference in my GeForce 2 GTS performance when I had to have the AGP clock to 83MHz instead of 66MHz (due to 133MHz CPU in BX chipset mobo). That's not to say that FSB overclocking is useless, just a real-life example.
What you were saying about programs being stored entirely in the CPU's internal memory cache. Don't count on it. If anyone were thinking that this would be a major factor, run some performance monitoring software specifically for the CPU (I can't find the software I used to use anymore, I'm going to have to email a mate to find out its name and download it again :-)), then run the apps you generally use to see what kind of cache hit successes you get.