Expected network throughput on Mediator?

[Oldsmobile_Mike]

In my own tests I found that 2.2 MBytes/s should be possible if you don't actually use the data transmitted. In reality with a web browser or an ftp client using the data transmitted you might get more than half of the 2.2 MBytes/s, and that's still a lot more than the Zorro II Ethernet cards can deliver. The X-Surf 100 is the fastest Amiga Ethernet card there is.

Probably a dumb question, but how is it that a RapidRoad connected to an X-Surf 100 in ZIII can get up to 6-7MB/sec (source:  http://www.fitzstevesamigaworld.co.uk/?p=253 ), but Ethernet only 2.2MB?  Is it all just protocol overhead?  Seems like 2.2MB/sec isn't a bus limitation if RapidRoad can do so much faster through basically the same pipeline.

[olsen]

Probably a dumb question, but how is it that a RapidRoad connected to an X-Surf 100 in ZIII can get up to 6-7MB/sec (source:  http://www.fitzstevesamigaworld.co.uk/?p=253 ), but Ethernet only 2.2MB?  Is it all just protocol overhead?  Seems like 2.2MB/sec isn't a bus limitation if RapidRoad can do so much faster through basically the same pipeline.

I know very little about how USB actually works, so this is a bit of speculation.

I reckon that the differences come about due to how much work the USB hardware does (as opposed to how much work the CPU has to do in addition to processing the incoming/outgoing data on Ethernet), how much freedom the implementors of the USB stack have (as opposed to the abstract "one size fits all" approach which the Amiga SANA-II network driver standard imposes) and how complex the underlying data transmission protocols are.

The type of Ethernet hardware we have to use for the Amiga is basically a data transport device, which sends or receives data which then has to be processed by software that, for example, has to verify or recalculate checksums, pick the structure of the data apart and make sense of it. This is by design, and not a side-effect. The CPU not only spends time on looking at the data, it also has to copy that data around 2-3 times between the network device and the application software using it. This adds up quickly.

In addition to shuffling the data around that comes and goes through the Ethernet device, the set of protocols which govern how the data flow is adjusted to the available bandwidth, and which detects and handles retransmission of missing data, is performed by the CPU as well. These protocols add a substantial layer of complexity, which is required by the fact that the data has to be able to travel a long way through several intermediate systems if necessary. So there is a lot of slack and elastiticy in these protocols to provide for this robustness, and that is just the opposite of what you would want in a fast data transmission system.

USB only has to cater for a local data bus which attaches peripheral devices such as hard disks or mice to your computer. There is no need to support mice connected to your computer by 10 km cables  Getting the data where it needs to be is also a lot simpler: the source/targets are all located on the same bus, whereas for TCP/IP the source and target could be anywhere on the local network or even the Internet.

Also, USB is likely designed so that the heavy lifting (more than just the basic work of doing error detection, retransmission, etc.) can be handled completely in hardware, with the computer tending only to the mandatory operations, such as handling device detection/removal, changes in the topology of the setup, and of course exchanging data between the computer and whatever is dangling out there.

Because there is nothing like the SANA-II networking standard for the USB stacks, and which today forces Ethernet hardware drivers to ignore all technical advances of the past 25 years, the USB stacks available for the Amiga can talk to the underlying hardware in any manner which they see fit, and which is likely best suited to the task at hand.

Now if you hooked up an Ethernet device to your USB setup, you might see that the overall throughput won't be in the same league as the basic USB data throughput. It would probably be a lot worse than what the X-Surf 100 can do all by itself.

[Oldsmobile_Mike]

Now if you hooked up an Ethernet device to your USB setup, you might see that the overall throughput won't be in the same league as the basic USB data throughput. It would probably be a lot worse than what the X-Surf 100 can do all by itself.

Thanks!  Okay, that makes sense.  I actually was assuming the opposite... that Ethernet could be handled more in hardware, so you've given me something new to ponder.

I noticed Amazon has quite a few USB-to-Ethernet adapters available for sale.  I googled terms like "speed of ethernet USB on Amiga", and came across this link, which seems to indicate in the charts at the bottom that a Deneb with USB Ethernet adapter is topping out at about 1MB/sec, with close to 100% CPU utilization.  So 2.2MB/sec beats that, any day!  

[Heiroglyph]

It can be dangerous to change the memory usage settings for the TCP/IP stack. Given enough traffic at a sustained high rate, the TCP/IP stack may end up consuming most available and usable memory (with respect to fragmentation, etc.).

I first saw this with AmiTCP V4, when I tested it with the X-Surf 100, and given that the underlying TCP/IP stack architecture is the same, right down to how the memory management works, Roadshow must be susceptible to the same issues.

Did you read the entire thread on which you found the tweaking information (there's some information on this subject in the Roadshow documentation, too)? There is more information to be found there, including how fast the X-Surf 100 can be expected to be, and under which circumstances.

I read the thread and the documentation carefully, although I may have missed something.

In my own tests I found that 2.2 MBytes/s should be possible if you don't actually use the data transmitted. In reality with a web browser or an ftp client using the data transmitted you might get more than half of the 2.2 MBytes/s, and that's still a lot more than the Zorro II Ethernet cards can deliver. The X-Surf 100 is the fastest Amiga Ethernet card there is.

You do realize that I don't have an X-Surf 100, right? I'm asking about an A4000 Mediator tower with Realtek 8139D 10/100 card.

I have never seen anything stating what can be expected from a Mediator like mine until I found Matt's netio output that I listed above.

[olsen]

I read the thread and the documentation carefully, although I may have missed something.

There are two aspects which I hope to have made clear in the documentation: 1) Roadshow ships with reasonably safe defaults which do not necessarily provide best performance and 2) tweaking Roadshow to improve performance is possible and recommended, but it has its limits and drawbacks.

You do realize that I don't have an X-Surf 100, right? I'm asking about an A4000 Mediator tower with Realtek 8139D 10/100 card.

Yes, I am aware of it. However, the limitations of both the X-Surf 100 and the Realtek card that can be used with the Mediator are sufficiently similar: it's not the network interface, but what surrounds them

TCP/IP network performance on the Amiga is limited by CPU speed, memory performance, network interface performance and TCP/IP stack architecture.

With the older Zorro II based Ethernet cards the limitations imposed by CPU and memory performance were not the bottleneck: the Zorro II bus and the capabilities of the interface (10 MBit/s) were. The TCP/IP stack architecture's limitations were practically irrelevant because the Zorro II bus and the 10 MBit/s Ethernet masked them. With a sufficiently fast CPU and high performance memory you could squeeze as much out of the 10 MBit/s Ethernet interface as the hardware allowed: I've tested this on an A3000T with an 68040 WarpEngine, for which Roadshow delivers almost 10 MBit/s with an Ariadne card.

With the X-Surf 100 and the Mediator the situation changed. The network interface no longer was the bottleneck, but the TCP/IP stack architecture became the limiting factor. All Amiga TCP/IP stacks which trace their lineage back to 4.2/4.3/4.4BSD Unix use intermediate buffers to move data between the application software and the network devices, and that involves up to three copying operations before the data is where it is supposed to be. This impacts the "CPU time budget", restricting how much data you can actually consume or produce in application software. On my test machine (that A3000T with the WarpEngine) I managed to obtain a raw transmission rate of 1.7 MBit/s with both Roadshow and AmiTCP Genesis, at 100% CPU load. If the CPU were faster, the transmission rate would have been higher.

I have never seen anything stating what can be expected from a Mediator like mine until I found Matt's netio output that I listed above.

I ported the netio tool natively to the Amiga, and rewrote the older tcpspeed tool (written by Holger Kruse for Miami/Miami Deluxe)  so that their measurements would be more accurate, and to cut down on transmission overhead. When I was testing the X-Surf 100 I wanted to be able to reproduce the netio test results I had read about. It turned out that the netio test results vary greatly, up to 40% between subsequent runs, so I would advise caution if you wanted rely exclusively upon them.

If you are using the original netio version, you are using ixemul.library for network operations, so there could be a bit more overhead involved than would be necessary. The netio port I made runs fine without depending upon anything other than bsdsocket.library.

If you are curious how these tools are measuring performance, please contact me via PM. I could send you the tools, with source code, if you like.

[duga]

Well, 100 Mb/s (megabits per second) equals 12.5 MB/s (12.5 megabytes/sec) with no overhead (error checking,mostly), but only about 50% can be used, so 50 Mbits/s data transfer =~ 4.7 to 6.0 MB/s (max).  Now you are getting < 1 MB/s, so there is a bottleneck.  The Buster chip limits the estimated 150 MB/s Zorro 3 speed to about 13.5 MB/s and the limit on Amiga to PCI card in a Mediator seems a max of 12 MB/s.  Jens of icomp.de says,  "Roadshow is a commercial product still available today, but since it does not reach the speeds that AmiTCP/Genesis can reach" but I can't find results of his testing.  He does state elsewhere, "Transfer rates of over 1600kBytes/s [1.6 MB/s] have been measured with the X-Surf-100. Please note that the classic Amiga architecture and the current TCP/IP stacks for the Amiga don't allow saturating a 100MBit Ethernet link. Performance in your system may be slower due to CPU performance and programs running in multitasking."

I think that somewhere in there is your answer.

You can reach 94-97 Mbps on a FastEthernet interface. But you won't reach that on your 'Classic' Amiga of course.

[danbeaver]

Oh, I didn't know that

[Oldsmobile_Mike]

You can reach 94-97 Mbps on a FastEthernet interface.

Sorry dude. I'm gonna let Bill & Ted take it from here.

[kolla]

Nothing bogus about duga's claim, not sure what you aim at Mike.

[danbeaver]

Well, at least the 94 Mbs is reachable in a closed laboratory setting between two PC's with no hubs and using ttcp (Test TCP) provided the "sender" only sends and the "receiver" only receives.

But, to the best of my knowledge we were talking about Amiga and their Ethernet cards and throughput over the Zorro 2/3 bus, multitasking, error checking and the like.

[kolla]

Well, at least the 94 Mbs is reachable in a closed laboratory setting between two PC's with no hubs and using ttcp (Test TCP) provided the "sender" only sends and the "receiver" only receives.

Not at all, you can reach those levels also with NFS across LANs, most equipment today is gigabit anyways. And full duplex means that you can both send and receive without one affecting the other.

But, to the best of my knowledge we were talking about Amiga and their Ethernet cards and throughput over the Zorro 2/3 bus, multitasking, error checking and the like.

Your knowledge is failing then, what he wrote was:

"You can reach 94-97 Mbps on a FastEthernet interface. But you won't reach that on your 'Classic' Amiga of course."

[Duce]

Those speeds are easily obtainable with modern hardware, no "test environments" needed, as Kolla says.  I do it every day of the week at those speeds and higher over my gigabit network to my NAS box using a $200 L2 managed switch, no rocket fuel or magic involved.

[Oldsmobile_Mike]

over my gigabit network

He's saying he gets 97Mbps throughput on his Fast Ethernet (which is 100Mbps, not gigabit) network. That would mean he's getting 97% efficiency with zero protocol overhead, etc. I'm calling BS on that. But regardless, TL;DR, off-topic for the thread, carry on folks.

[kolla]

Sheesh, FFS!!

He said FastEthernet *interfaces*, not network, stop twisting this around! Today equipment is typically geared toward Gbit speeds, and yeah, 97% efficiency on 100Mbit interfaces is quite doable - nowhere did he mention protocol overhead, because that is irrelevant when it comes to measing efficiency of *hardware*. You chose to use a slow protocol with lots of overhead, that your effin problem!

[Oldsmobile_Mike]

He said "You can reach 94-97 Mbps on a FastEthernet interface". FastEthernet is a technical term for a specific type of connection that I'll be damned if you will ever get 97 Mbps real-world throughput on.  Not on it's best day, with all your prayer beads in a row.  But yeah, whatevs, really.   "Xerox" used to be a specific thing, too.  Now it just means "make a copy".  Google used to be a brand name.  Now it's just a verb to say "I did a search on the Internets".  Blah blah.  Have a nice day!  

[kolla]

Well, I'm sorry to see you strugling with your network, as well as your grammar.