If you want a machine just for whdload, then i am sure you will be just fine with a desktop. You will need to get a better psu for it though. And a CF card work just fine in an Amiga, as AmigaOS dosent write stuff to the HD every other second. A windows pc will constantly write to the disk, which is why CF is not recommended on a pc.
If costs are the reason for not getting a towerized box, then i think you are kinda wrong as i have seen quite a few cheap towerized amigas here in EU on auction sites. I myself bought one for around $90. It came in a RBM case and had a cdrom, 4gig hd and a blizzard 1220/4mB so the specs are not too great, but it works fine with nearly all whdload games and makes great machine for those purposes.
Building your own might be quite a bit more expensive though.
But still, if i could get one in a orginal desktop with similar specs for same price, then i would have gone for that.