I did the same thing a while back. I bought a G4 533 "Digital Audio" before the mini was announced. I love both the machine and OSX. Although usable, the OS does feel a little sluggish on this machine even with plenty of ram.
I would recommend following through with the processor upgrade like you mentioned. I still haven't, and I think I would use the machine a lot more if I did. I have to admit it's a beautiful OS though:-D.
Jeff