Depending on the CF adapter you are using it may actually be two devices. Some of them have the pads for a 2nd CF slot but its not used, this still treats it as 2 IDE devices when installed and a lot of CD-Drives will refuse to work on the same channel as it. You most likely need to find one that's just a single CF slot and not just one that was built without the other soldered on.
The AmigaKit ones are two devices with only one soldered on. They require the CF adapter on one port and the CD on the alternate channel with that IDE buffered adapter AmigaKit sells.
Ye even if you put the CF as master and the CD-rom as slave this happens.