I connected my A500 to my PC which dual boots Win XP or SUSE Linux 8.1 via serial port to com1. I have only tested the serial connection under WINXP, 98, and 95. I used standard serial cables as well as a NULL MODEM box. What OS are you using on your Amiga? If it is Linux, then ignore the rest of my post.
On the WINXP machine I can connect using Hyperterm at 57600 Bps, 8N1, hardware control on,using ZMODEM protocal.
On the A500 (WorkBench 2.1), I connect using Terminus from Aminet
Terminus 2.0e. On a stock A500 using the serial.device the fastest reliable connection via a normal serial cable I found to be at 28800 bps. I was able to achieve 57600 by using BaudBandit.device as a replacement
BB.device aminet.
I have not yet got around to testing the connection in Linux, as I'm very new to the OS. perhaps you can tell me what Linux/Amiga software you are using so I can recreate your problem myself, and try to find a solution.