It will be a tight squeeze to fit a 3.5" hard drive inside the A1200 but it is possible. You will need either a 2.5" to 2 x 3.5" IDE cable or a 4-way buffered IDE interface (you can get them from AmigaKit). You would need one anyway to connect a CD-ROM drive (for installing OS 3.9).
When you install OS3.1 (which you must do before installing 3.9), you should only partition and format the first 2GB of space on the drive, then install 3.9, then use the HDToolbox from 3.9 to partition/format the rest of the space. (AmigaOS 3.1 and lower does not support hard drives larger than 4GB.)