Whenever I remove a battery I always use vinegar to neutralize the acid, I apply a generous ammount of vinegar and use a toothbrush to sort of brush off any lumps of battery acid. Then flush with water. I then use a spray originally intented to clean radio control car electric motors motors, which then displaces any water and evaporates leaving no residue. Then let dry for 24-48hrs just to be safe.
Using this process I have never had a problem.
Most of the time I replace the original battery with a GP Nimh, although you do not have to do this.
