@Hyperspeed
The drive doesn't actually mind re-reading and writing the same old spot over and over again. The platter stack spins anyway and the heads don't (really) have to move.
However, it does mind the read/write head being moved like crazy all over the platter when fragmentation is high - just listen to the sound the drive makes: this is certainly not exactly conserving its life.
Depending on the usage of the drive, overdoing on defragmentation might put more stress on the drive than living with fragmentation as you've pointed out. If the drive has a relatively high stress level (e.g. Windows system drive), it's good to have the static files defragmented to reduce the stress on the mechanics. If your drive works as an archive for text documents and such, there's no point in defragmenting it over and over again. Usually it's a good idea to separate the system files and temporary data on different partitions, so the latter cannot fragment the former. (Of course, only putting new files can get them fragmented, so if you don't change any of the files regularly needed, there's nothing to be feared.)