As far as I have read, if you haven't got the 3.1.4 ROM, the OS loads the necessary extra parts from disk on startup. It results in a slower startup but that's it.
I've read that OS3.2 does the same. The caveat is that if you put in the 3.2 ROM's, 3.1.4 won't start when you switch on to initially install 3.2. You need to leave the 3.1.4 (0r earlier ROMs ) in, install 3.2, then replace the ROMs.
