Yanking out is barbaric, but it works. Really recommended way to do this is to gently pry the battery up and out of the PCB using a needled nose pliers and soldering iron. 15-30watts iron that is - NOT a gun. This way, you can re-install a newer battery or better yet, one of DJBases's lithium/diode hack more easily, cleaner and professionally.
