I run OS 3.2 in emulation, so in my case it does not touch the actual hardware. I am using the OS 3.2 ROM file as supplied on the OS 3.2 CD. OS 3.2 requires more RAM than OS 3.1 does. Not sure if this is an issue for you.
As I recall, OS 3.2 works best with the OS 3.2 ROMs.
If you want to go back to OS 3.1 ROMs, I would suggest going back to OS 3.1 entirely. I never had 3.5 or 3.9 so I don't know if they used 3.1 rom or tailor made roms.
---
redfox