I think lithium battery hack looks much better than cordless telephone. Its smaller.. besides, what if the new rechargeable battery leaks too.
If you do the lithium batt hack, use Schottky diode like someone here said, its better (though an ordinary one will work too)