You'll find the glue it is attached with doesn't really soften much as it gets hot.
Crazy as it sounds, Speel is right - but you have to be very careful. The glue is brittle when cold and you can generally remove a heatsink with a chisel. I removed my old 040 card's pathetic heatsink and fan this way.
BUT: I wouldn't try this on a gfx card heatsink! The chances are that the BGA connections would come undone before the heatsink.