This is gonna sound silly but could it be the serial driver is compiled for the 68020 and as such simply doesnt work on the A600's 68000? (16bit vs 32 bit)
I would also try connecting a normal serial device, an old mouse or modem for example and see if that works. (Although as you wrote a program to open the serial port and even that returned an error then its doubtful its the usb to serial adaptor or cable)
