The way I understand it is that this is because compressed tunes sell rather better than uncompressed ones. It is not a problem of the medium; it is a problem of targetted marketing. Since LPs are manufactured in such small numbers nowadays, you can afford a gray beard who still knows how to make a recording which sounds good, and not just a recording the suits think sounds good. But this is all a bit off topic.
Yes, you are completely correct about it not being a limitation of the cd format. Most cds that were made in the early days of the cd format was infact not compressed like this.
I wish they could just implement a button on all amps/recievers, so that people can choose themself if they want loud and flat sound instead of full dynamic range.
But the result is still the same... The majority of commercial music produced today, just simply sound better in the LP format in most cases.