You may want to do some reading on "Serial" "Parallel" and "SCSI" ports, as there seems to be some confusion here between the three.
"baud rate" is (in simple terms) the data transfer speed through the serial port - iirc Silver Surfer has something like 460800 bps baud rate, whereas the amiga in-built will just about manage 57600 on a good day (and this could be an overestimate). So yes, the silversurfer would be significantly faster.
Either way you cannot attach a parallel scanner to a serial port. Well, maybe you can given the right cable, but the burning smell might make you want to unplug it pretty fast.
Also, probably not a good idea to connect a scsi scanner to a parallel port either.
