X-Surf 100 - Best TCP/IP for Speed?

[olsen]

Currently I own Ariandne with MNI drivers in Miami, and working for the big speed increase...

Getting funds together to buy this awesome card and was wondering:

Is roadshow the fastest for the card?
What is the best TCP/IP stack for multitasking and system performance?
Anyone using roadshow with this card?


Appreciate the feedback...

Funny you should ask, as I recently did a little testing to answer just that question. Courtesy of Jens Schönfeld, who loaned me a sample for testing. For the record, the "X-Surf 100" is the fastest Amiga Ethernet card ever made (if used in Zorro III mode, and connected to a 100 MBit/s or 1 GBit/s switch). You can quote me on that.

First thing, any TCP/IP stack you can buy today is going to be faster than "Miami", even with MNI drivers. That goes for both "AmiTCP Genesis" and "Roadshow".

If you have an A3000 or A4000 then the "X-Surf 100" will send and receive data faster than your Amiga will be able to process. With the "Ariadne", it's the other way round: the computer is faster than the Ethernet card. The "Ariadne" limits how fast you can send and receive data.

But that's not the case with the "X-Surf 100". No matter which TCP/IP stack you use ("Miami" probably being the exception, because its SANA-II network support limits how fast it can go), you're probably not going to see much of a difference between "Roadshow" and "AmiTCP Genesis" in terms of speed: the "X-Surf 100" is the great leveler.

"Roadshow" plays to its strengths when it comes to make the most of available memory and CPU power, especially if you don't have a high end Amiga or a high end Ethernet card. If you have lots of memory, and a very fast CPU (say, 68060@50MHz), "Roadshow" won't provide more punch than "AmiTCP Genesis", when using the "X-Surf 100" in Zorro III mode, and you transmit data under full load. I do believe, however, that "Roadshow" does well enough with modest means, i.e. you don't have to throw a lot of memory at it in order to get it up to speed.

What I found interesting is how much the CPU power matters in terms of TCP/IP stack performance. In my TCP transmission tests (in the local network) I got about 1.700 MByte/s out of the "X-Surf 100", on my test machine which I used for benchmarking "Roadshow". (Note that I did not conduct tests between the A3000T and the Internet, since measuring network performance for this application is very hard to do reliably. You're not just pushing data around between a source and a destination, but a lot of intermediary systems, too. Talk about reproduceability...)

Both "Roadshow" and "AmiTCP Genesis" ran about exactly as fast in my tests. My test machine is an A3000T, with a WarpEngine 68040@40 MHz. I read about a CyberStorm Mk II, with a 68060@50 MHz, getting about 2 MByte/s out the "X-Surf 100", and I believe that's a realistic figure.

Note that when the "X-Surf 100" is transmitting data at this speed, your CPU load will be at 96%, if not higher. This is because no currently existing Amiga TCP/IP stack supports offloading tasks such as checksum calculation/verification or data transfer to the networking hardware. Whenever an Ethernet frame enters or leaves the "X-Surf 100", it will be your computer's CPU pushing it along. Whatever CPU cycles are still left, while the "X-Surf 100" is running under full steam, so to speak, your networking software may put to good use. Both the TCP/IP stack and your networking software will be competing aggressively for the CPU.

Because so far Amiga networking hardware was very slow, we never had a case of the CPU power being the limiting factor when it comes to network I/O performance. Bottom line is, if your CPU is very fast, then the network I/O performance will be high. If not, then performance will be proportionately lower.

In my personal tests of the card and its driver software, I found that it was almost twice as fast as the original "Ariadne", which is the fastest Zorro II Amiga Ethernet card I know of. With a high end Amiga configuration, I reckon that it the "X-Surf 100" will be a bit more than twice as fast as the "Ariadne". I think sustained transmission rates of more than 2.2 MByte/s may be possible.

That's as much as I found out, during my own personal testing (your mileage may vary). If anybody's interested in doing their own test, I'm happy to share the source code of the benchmarking programs which I used for testing, and the test methods.

[olsen]

@Olsen

Thanks!

I'm using a 3000D with a CS MKIII 060-50.

Looks like you've got the right gear to make the most of the X-Surf 100 then.

I will use the easynet front end and hope they update it to work with lpr.device in the near future for printing.

Well... should you ever get curious, the free "Roadshow" demo will still be available for evaluation. Plugging in "lpr.device" in the services configuration file should not be too hard. Come to think of it, I'll make a note to add a sample entry for it.

One issue I had with the Ariandne and my Delfina Plus, is that if I copied large files from my NAS and played "insane" mp3's, (If I copy smaller files and/or played normal mp3's there is no issue) I would get stuttering playback.  The Delfina Plus only uses 2% CPU, so I thought it was bus related.  Perhaps this may be CPU, and will not go way with the X-100.

It probably will not get worse either. How you describe it, it seems to me that the issue may be with the size of the mp3 decoding buffer.

If the file to be played is small, it will transfer quickly over the network while the player can either still keep up with decoding, or will decode it just in time for the network access not to have a negative impact on it. But if the file is large enough, decoding, playback and reading more data off the network will happen in parallel. Because the decoder and the TCP/IP stack are competing for the CPU, playback quality inevitably suffers.

I had the same problem with the "Concierto" add-on for the Picasso IV. Because the "Concierto" hardware was interrupt-driven and could not use DMA (on top of that the sound chip had its own quirks), playback always consumed CPU time. Playing back mp3 files on the "Concierto" left little CPU time to other tasks. You could either listen to the music or do something else, but not both.

Now if you had a larger decoding/playback buffer, it might be that the player will be able to feed audio data to the Delfina Plus more easily while it's reading more data from the network. Increasing the buffer size, if that is possible, would be the classic remedy for this kind of problem.

[olsen]

What test do you recommend for calculating speeds of the cards?

I use "synthetic" tests to gauge performance, which as danbeaver pointed out, have little relevance for real world applications. Still, these tests can be easily repeated, the results are reproduceable and comparable.

For my recent tests, I ported the NetIO program to the Amiga and reworked the TcpSpeed program, which I believe was written by Holger Kruse, who also wrote Miami/Miami Deluxe. Both programs require Kickstart 2.04 and a plain 68000 Amiga (i.e. they both work on an A600).

The reworked TcpSpeed program tries to measure the throughput more accurately by resorting to timer.device measurements and using floating point numbers to calculate the figures.

Both NetIO and TcpSpeed perform data throughput tests, i.e. they transmit fixed amounts of data between a sender and a receiver and measure how long it takes for the last byte to arrive. From these figures they calculate the average data throughput.

For the tests to work you need two computers. One acts as the test server, the other as the test client. The test server in my case was a Mac Mini, and the client was an A3000T. Both were connected by a Gigabit Ethernet switch.

Programs and source code are available from me only for now. I have not yet uploaded them to Aminet. If you are interested, please contact me via PM.

[olsen]

OK, installed the x-surf 100.

Using Miami right now and it is barely faster than the Ariandne.

Ariandne with MNI gives 155,000 cps and 25% cpu.
X-Surf with SANA2 gives 170,000 cps and 15% cpu.

Oh well.  Hopefully when I figure out how to install AmiTCP with Easynet, the results will double...

The Miami SANA-II support leaves much to be desired in terms of performance. Considering how fast the X-Surf 100 actually is, seeing that with Miami it runs faster than the Ariadne, even when using SANA-II drivers, should tell you something

[olsen]

Is the difference DMA usage? it seems like a significant difference.

Neither the Ariadne nor the X-Surf 100 support DMA.

If X-Surf 100 did support DMA, and the TCP/IP stack would so, too, then the figures would be much, much higher.

The problem here is that the Miami SANA-II support is not as good as it ought to be, and obscures how fast the X-Surf 100 really is.

[olsen]

This just tells me that AmigaOS need a new network driver infrastructure..

It's not so bad. The thing that's missing is a set of data structures to accomodate modern networking hardware, allowing the driver to offload the copying operations which currently slow down 100 MBit/s Ethernet traffic.

The remaining parts of the SANA-II standard do hold up pretty well, even after more than 20 years.

[olsen]

It already has that in OS 4.1.

Not really. The network drivers for OS4 still use the same SANA-II standard as the TCP/IP stack (which is the PPC version of Roadshow). Everything SANA-II revision 5 can do can be done on 68k Amigas (and of course MorphOS), too.

[olsen]

Miami has always been known to be slow. It was my understanding that the key checking and such holger had in the code really slowed it down also.
roadshow on the same hardware is usually good for quite a speed increase.

That sounds like a technically convoluted approach to the task. If it slows down the main task of the software to a degree where it really suffers in terms of performance, something is deeply wrong with the implementation.

My best guess is that the I/O code in Miami just didn't get revised/reworked as it would have been over the years. Holger went out of business before that work could have taken place.

Maybe it is time for SANAIII  olaf?

SANA-II is solid, but it doesn't support the kind of networking hardware well which has been around for "only" 15-18 years now. It was designed for the type of Ethernet gear which was available in 1987-1990, and that it supports well.

In my opinion "all" it takes to make SANA-II work better with modern Ethernet hardware would be to standardize support for offloading work (copying, calculating checksums, verifying checksums) to the hardware. This has been tried before, and it requires support both in the driver and the networking software which uses it (e.g. a TCP/IP stack such as Roadshow).

Given that the Amiga market has slowly turned into a hobbyist affair, there just doesn't seem to be as much fun in working out the details of getting samples of how different the hardware is that needs to be supported, finding common ground, putting together an example implementation, reviewing the results and then deploying it.

[olsen]

The Amiga network drivers suffers from inefficient buffer handling scheme and proper support for promiscuous and multicast modes. On top of that there's some version requirement too.

It's a mess!

Perhaps it's old age setting in, but I'm not so pessimistic about SANA-II. Having worked with it for almost 15 years, I may have gotten used to the quirks.

Inefficient buffer handling, yes, that's the big one. However, it only hurts if you use it with a modern NIC

Proper support for promiscuous mode is missing, i.e. there is no SANA-II command for changing or querying it. The best one can do is to use the S2_READORPHAN command and hope for the best.

The multicast support sort of works, and you even have a choice of two different commands to do the job (ha!). Mind you, neither of these will allow you to receive multicast packets from arbitrary addresses. They either have to be within a certain range, or you have to use one and only one address to listen to.

I wouldn't call this a mess. Unless you badly need this functionality (multicast comes to mind), I'd say the vast majority of users can squeak by without realizing what is missing. The limitations of the buffer handling are the most annoying thing.

What do you mean by version requirements?

[olsen]

Only working on some OS versions.

In which context? I can't say I have heard about this before. Except perhaps concerning the A2065 driver, which for many years was quite poor in terms of robustness and speed.

I just think it would be a serious improvement to get a clean and efficient API.

What do you have in mind, specifically?

[olsen]

Weird, just plain weird.

EasyNet is slow only getting 30,000 cps.  Tried increasing the buffers to 4096 with no change.

Careful with these settings, they don't necessarily have the impact which their names may suggest. Change the wrong combination of settings, and you'll render your TCP/IP stack "combat ineffective".

Do you have the latest x-surf-100.device installed? The version I ran my tests with was V1.11, released on September 16: http://wiki.icomp.de/wiki/X-Surf-100

The archive which contains the driver also contains an example "AmiTCP.config" file which turns up the memory dials to 11 and which will improve data throughput. Careful though, these settings don't necessarily work robustly and may have side-effects: they throw a lot of memory at the task. Less memory may still work, but so far nobody has suggested how much less.

The only thing I can even think of, is that I had to switch to opendns for dns to even make EasyNet work.  Road Runner DNS will not work in EasyNet but work fine in Miami.  Could this impact download speed?

Not unless you're constantly performing DNS lookups, which I consider unlikely. AmiTCP doesn't have a DNS cache (Roadshow does), so if DNS lookups for the same name are frequent, then it will spend considerably more time performing them.

DNS lookups failing is usually a sign of network trouble. A DNS lookup is usually performed by sending UDP packets to the DNS server and then waiting for it to respond. If the packets are corrupted during transmission, or get lost, the lookup may be repeated later. But there is a long timeout before the lookup is retried, and if that doesn't work out again, things will get unpleasant.

As a rule, you should use your ISP's DNS server if you can, because the likelyhood of losing packets is smaller with them than with a different DNS server, e.g. OpenDNS or even Google: the packet don't have to travel very far.

I can perfectly understand if you don't want to use your ISP's DNS server, though. Mine is known to produce responses which do not reflect the original queries. Because DNS security is so brittle, I rather run my own DNS server at home and use that instead.

The next best thing is to have your gateway router do the DNS lookups, cache these, and make your Amiga use it. Find a trustworthy DNS server and stick with it (try http://www.opennicproject.org/).

For a while my cpu was spiking to 90%, but now it's back down to 15% with either Miami or EasyNet.

Miami SANA2 peeked at 200,000 this morning and 15% cpu.

To me this looks like you might be dropping packets.

[olsen]

Tried the AmiTCP.config file from the archive and it only got me to 60,000 cps.

That should have made a difference. But, well, it's not the change that I would have expected. Bummer.

Also set up AmiTCP with the suggested DNS server, no difference.

I don't know how this works with AmiTCP, but I'd suggest you have a look at what the TCP/IP stacks says to how many errors cropped up during the transmission. On a Unix system that would be done through a shell command called "netstat", e.g. "netstat -s".

The only things I can think of is:
1.  When I download the update via easynet, is seems extremely fast.  Best guess at least 250000 cps.
2.  I haven't changed my router away from road runner dns yet.  Maybe it is conflicting with amitcp settings and the opendns.

If you select a specific DNS server pair in AmiTCP then this is what will be used, regardless of what the router wants to use. Your ISP may or may not manipulate outbound DNS queries and the corresponding results. It's technically possible, and if you're living in the UK, it might actually happen.

3.  Raw copies accross network seem much faster.  For downloading from aminet, Miami is 4 times faster (again pointing to DNS)

No, this probably isn't it. DNS has one responsibility only, and that's mapping human-readable names such as "www.amiga.org" to IP addresses, such as 50.87.149.192 (and the other way round, too).

Your networking application software (ftp client, IRC client, web browser) will start by querying the IP address associated with a name, and once it knows the answer, will proceed to open a connection to the IP address.

Once the connection is up, the web server or ftp client can start transmitting data. At this point your choice of DNS server is probably completely irrelevant for the performance of the data transmission. "Probably" because in certain cases network services may be used to pick the closest available server, based upon where your DNS query originated. This is how, for example, content distribution networks (e.g. Akamai) may optimize delivery. If you're unlucky, you might end up getting the IP address of a server which is not optimal for your place in the internet.

What else could be slowing things down? If you're sharing the internet link with somebody else, it could be this person's network traffic getting in the way. If you live in (sort of) a backwater town (like I do), it could be the poor quality of the broadband connection which due to signal attenuation restricts how fast you can go. In my case, on a bad day I've got 6.5 MBits/s (downstream), and on a good day it's 9 MBits/s (both are pretty far from the advertized 15 MBits/s). Finally, your ISP could be limiting your traffic on purpose.

Do I need to change my router over also or can amitcp access a different dns with out issues?

Whatever you set up in AmiTCP counts. Unless your ISP filters the DNS packets which pass through your network and chooses to alter it.

[olsen]

I have actually come across some routers which defy this rule.  Some older Engenius routers (I stopped using the damned things, so I cannot speak about newer units,) some Belkin, and a couple of D-Links were culprits in breaking Windows Server DNS services.

Now that really sounds scary.

I'm surprised that anybody would be so reckless as to muck with DNS queries within a gateway router. There's little to be gained, but it doesn't take much and possibly the most important service your local network needs will suffer from it.

DNS is surprisingly brittle. Let's say you have three DNS servers in your network for redundancy, in case one of them should fail. The way you set up the client computers which rely upon the DNS services is to enter the IP addresses of the DNS servers in a configuration file. Problem is, the DNS queries are usually made in exactly the order you entered them.

If the first server should become unavailable, the query will time out and the next server will be used. Because so many services which perform DNS queries do not cache the results, having the query time out because of a failed server will mean that they will repeat the queries over and over again. Net effect: the entire client network will become stuck because the first DNS server became unavailable. How do you avoid that? I'm not sure that this is a solved problem

Anyway, so far I've been lucky with my choice of gateway routers at home, and even rolled my own OpenBSD-based firewall/PPPoE solution when the old gateway router I was using at the time failed.

I don't know about the Belkin gear, but I know of at least one D-Link router which we ended up replacing at my company. I wouldn't buy one of these again. Same goes for Linksys gear.

I used to use NetGear and ZyXEL gateway routers, which at least 4-6 years ago used to be quite decent. The German-made Fritz!Box routers (don't laugh, it's the actual brand name), which run on Linux are very decent, too.

I was able to show that the routers were intercepting DNS packets.  I some cases the inbound was blocked, forcing the server to use the router for DNS.  In other cases, outbound DNS packets were being usurped, and this caused huge issues with EDNS queries as none of these routers support EDNS so queries would be delayed, broken in sequence, or in most testing simply dropped.

It's easier to tell what's happening with your queries if you have access to an outside DNS server's logs.  But it's not impossible otherwise.

True horror stories... You have to be able to trust your gateway router to do what it's supposed to. Kudos for actually proving what was going on. That must have been a real nightmare, both figuring out that the router was up to no good, and finding a remedy for the problem (before tossing out the gear, I suppose).

[olsen]

You can't trust propietary stuff. Add to the mix the mandatory US goverment mandated backdoor.

Not sure if there is such a thing.

From what I gather, in most cases it would be unnecessary because the lack of robust security in the firmware is a perfectly adequate substitute which also provides for convenient plausible deniability: hey, that was a bug/typo/manufacturing test.

Getting a useful backdoor into the code which is not easily discovered or used by other people who are up to no good (poster child: wiretapping functionality in Ericsson switches used by Vodafone in Greece enabled and used by persons unknown; discovered by accident in 2006) is difficult. The complexity of the task will almost inevitably lead to the backdoor being discovered.

[olsen]

@Olsen

Installed Roadshow Demo.

Wow, wow, wow, and wow.  350,000 cps peek and always above 300,000.

Would that be local network traffic or internet traffic?

I still think that this figure is too low for the X-Surf 100.

I can get that much out of plain Ariadne on a foggy day with drizzle and light snow (in fact, in my tests Roadshow squeezes as much out of the Ariadne as you possibly could expect from a 10 MBit/s Ethernet card: more than 900,000 bytes per second).

The X-Surf 100 should peak somewhere above 2.2 MBytes per second, and unless your internet connection is very poor in terms of throughput, you should be able to get at least 1.5 MBytes per second downstream traffic out of it, probably more.

Ok, so now I need to test smb shares on the nas and samba.  Assuming those work, and you have yourself another customer.  Don't know what Easynet is having trouble with but, it simply will not perform, was just going to use Miami and hang out at 200,000.  Not any more, your product screams...

Well... I still think there's something fishy going on. It's nice to read that Roadshow performs better for you than both Miami and AmiTCP do under the same circumstances, but the throughput figures are far from what the X-Surf 100 is capable of.

Incidentally, the Roadshow "DEVS:Internet/servers" contains ready-made entries for smbd and nmbd, which are commented out. If you have everything installed in Samba:, then everything you need to do in order to give Samba a spin is to remove the leading '#' characters in the last three lines of the file.