Your specs are better than what I used (a1200 apollo 040, 32M ram). I do remember having to fiddle with the settings quite a bit...
Here are my settings, work on a basic install of OS3.9 (no boingbags), if you want me to look at anything else i didnt cover below, let me know.
X-Audio using paulaaudio.device
buffer: 5secs
prebuffer: 30 secs
priority: 10
mpeg audio 1.2 player config:
------------
paulaaudio.device
buffersize 256k
blocksize 32k
layer I/II/III all (samplerate 1:4) (quality: low) mono