When setting the AHI prefs, make sure to use a HiFi stereo++ mode. Increase the frequency to 48000 and the channels to at least 8. In AHI v4, increasing channels will decrease the overall volume - version 6 corrects this.
What are your versions of delfina.library and delfina.audio?
To upgrade to AHI 6, basically just replace the v4 files with the ones from the v6 distribution. I don't think there are any new ones.
EDIT: Also, which Delfina board is it?