Oh yeah, and it's worth noting that most JPEG files are actually JFIF files, which have different headers than EXIF JPEGs. Technically, the ".jpg" and ".jpeg" extensions are wrong. :-)
This is noticeable in many programs that expect JFIF formats. Load up an EXIF from a digital camera, and the picture will either show up blank or crash the program. We had this problem a lot on our Kodak workstations. I was furious at Kodak at first, but when I started doing a lot of image recognition in my web scripts, I started to realize just how many different JPEG "standards" are out there -- or really, the lack of them.
Yet another thing we can blame on "extensible" formats. I prefer an actual standard that doesn't change, because a bad standard is at least a standard. Metadata can be very, very troublesome, and is usually more marketing hype and feature-pushing than a real help.