uIP/lwIP for Amiga OS

[Colani1200]

Trev wrote:
If the A600 is taking longer than 1 second (de facto standard delay between requests) to respond to an ICMP echo request, then yes, something will get dropped.

I didn't test that with ICMP, but browsed the web pages and the dropped packets were increasing quite quickly.

[Trev]

I do know that with a single receive buffer, the stack doesn't process incoming requests quickly enough. For example, Internet Explorer ACKs the connection and sends the first HTTP GET in less than a microsecond. uIP hasn't yet processed the pending I/O request for the ACK and so never receives the first HTTP GET.

Using more buffers uses more memory, of course, so we're down to a memory/performance trade-off. The number of I/O requests to queue will be an option soon.

Trev

[Trev]

Multiple receive buffers have been implemented and a command-line option (BUFFERS) has been added. The minimum number of buffers is two--one for IP and one for ARP. Currently, the first buffer is reserved for ARP and the rest are used for IP. Each buffer consumes ~600 additional bytes of memory. Three or more buffers provide a decent level of interactivity.

The device I'm using (openpci_8139.device) sends "orphaned" frames to all pending S2_READORPHAN requests. I'm not sure if all devices do that, but I suppose it makes sense. Device driver authors have no way to determine the intent of a task or process when it queues multiple S2_READORPHAN requests. Anyhow, I was receiving multiple copies of frames after implementing mulitple receive requests. That lead to the split between ARP and IP requests. Reads and writes are still raw (the entire frame minus the ethernet trailer is copied to the buffer), but reads will only catch ARP and IP frame types.

EDIT: You know, a device also has no way of knowing how a process or task is going to use multiple ARP or IP requests. As devices are required to service requests for all callers as of S2R3, device driver authors would have to make a conscious decision to send one frame per CMD_READ request once per task or process and not the same frame across multiple CMD_READ requets. Not sure why the same decision wasn't made for S2_READORPHAN requests. More digging to do, I guess.

The PNG image in files.shtml failed to transmit because of a bug in the Perl script that converts content into C. The script was using read(), which wraps stdio. On Windows, CRLF sequences are converted to LF. Not a good thing for binary files. Replaced read() with sysread() to fix that.

Still have checksums to implement. I also have a few ideas about why certain devices aren't initialized properly.

[Trev]

A 680x0 specific checksum routine has been shamelessly stolen from RTEMS (uses a BSD stack) and modified for vbcc. Oh, the joys of open source software. Cycle-wise, it's about 250% faster than uIP's default routine (but only a pinch faster than the BSD optimized C routine). Being BSD-derived, the routines are very similar to AmiTCP's. I'm steering clear of AmiTCP code, however, since it's GPL. I'd rather stick with a BSD-style license that doesn't place any restrictions on use or distribution (short of copyright notices).

Toni Wilen has added a SANA-II device (uaenet.device) to the latest builds of WinUAE. Very cool, but it currently requires TAP-Win32. I'd rather see an implementation that doesn't require a bridged virtual network adapter. Microsoft Virtual PC does it (of course), and so does nmap. Regardless, cheers to Toni for all the hard work.

Still confused as to why some adapters configure and go online and some don't. For those that go online after being configured by another stack, e.g. Miami, the problem is obviously with my initialization code.

If anyone wants to take a stab at using sanautil (available on Aminet) to configure and bring a card online, please let me know the results. I'm particularly intersted in the output of "sanautil device status" before and after the card is configured and before and after the card is brought online.

Here's the compatibility list as it stands today:

ariadne_ii.device,43.12,no
cnet.device,1.9,yes
hydra.device,?,yes
ioblixether.device,37.14,no
norway.device,?,yes*
openpci_8139.device,1.2b4,yes
prism2.device,?,yes
uaenet.device,1.0 (1.4.5b13),yes**

* after configuration by separate stack
** acquires DHCP address; hangs on first ICMP echo reply

Trev

[Trev]

My checksum routine wasn't working 100% of the time, so I had to put together a nice set of test cases. Anyhow, it should be good to go now. Here's the current set of command-line options:

Code: [Select]


/*
 * Command-line options.
 *
 * Default: DEVICE Devs:Networks/openpci_8139.device UNIT 0 CONFIG DYNAMIC
 *
 * DEVICE
 *   Fully-qualified path to device to open.
 *
 *   Example: DEVICE Devs:Networks/openpci_8139.device
 *
 * UNIT
 *   Unit number to open.
 *
 *   Example: UNIT 0
 *
 * BUFFERS
 *   Number of buffers to allocate.
 *
 *   Example: BUFFERS 10
 *
 * CONFIG
 *   IP address configuration. Valid values are DYNAMIC and STATIC. If
 *   STATIC, the ADDR, MASK, and DEST options are required.
 *
 *   Example: CONFIG DYNAMIC
 *   Example: CONFIG STATIC
 *
 * ADDR
 *   IP address. Required if CONFIG STATIC is specified.
 *
 *   Example: ADDR 192.168.0.100
 *
 * MASK
 *   IP subnet mask. Required if CONFIG STATIC is specified.
 *
 *   Example: MASK 255.255.255.0
 *
 * DEST
 *   IP default gateway. Required if CONFIG STATIC is specified.
 *
 *   Example: DEST 192.168.0.1
 */


And a link to the latest revision:

http://www.spookysoftware.com/uip.lha

I still expect folks to have problems configuring and bringing online certain adapters. If anyone gets a chance to run sanautil as requested above or knows the ins and outs of configuration for a particular device, please let me know.

Apart from that, this revision should be more responsive--both uIP and the OS itself--than the last. I'm interested in ping response times for various Amiga configurations using a full-sized frame, e.g. ping -l 572 n.n.n.n.

For best performance, use three or more buffers (the minimum is two). After that, it's a memory/performance tradeoff. There's also a point where increasing the buffers won't help, as uIP only processes pending outbound frames twice per second (configurable in code). That's why the shtml pages are so slow--the data is sent periodically.

EDIT: Oops. Forgot to put a buffer size check back in. Binary updated. Note, the URL has changed.

Trev

[Trev]

Another little revision. I modified the DHCP client to randomize the transaction ID field per process and use the client indentifier option. Assuming your ethernet device supports it, this should allow you to run multiple instances of uIP on the same interface and still allow use of DHCP.

This would be required, for example, if you wanted to embed uIP in multiple applications and run them simultaneously. Each application would have its own IP address. This won't work if your device only sends an incoming IP packet to the next pending IO request. If it sends the packet to the next pending IO request per process, then you're good to go.

Caveat: I haven't implemented DHCPRELEASE, so keep an eye on your DHCP scope(s).

http://www.spookysoftware.com/uip.lha

EDIT: Also, Toni has switched from TAP-Win32 to WinPCAP for the new uaenet.device. Looks even more promising now, although there are some new issues to resolve. We all know it, but WinUAE is one awesome piece of software. Everyone involved in UAE's development over the last 13 years (has it really been that long? yikes) really deserves our gratitude.

Trev

[Colani1200]

Trev wrote:

Apart from that, this revision should be more responsive--both uIP and the OS itself--than the last. I'm interested in ping response times for various Amiga configurations using a full-sized frame, e.g. ping -l 572 n.n.n.n.

Ping times are usually between 55-60 ms on my A1200 030/50 with prism2.device (buffers 10). I noticed some strange behavior though: The first pings (looks like the number is related to the buffers?!) seem to be fired very fast. After that, they happen in the usual 1 second interval. When I ping for the first time, I also get a whole bunch of "ping: sendto: Host is down" messages before it actually starts working. The OS indeed seems to retain its normal responsiveness when uIP is running in the background, so that's a great improvement.

However, I have a problem with uIP on the A600. It won't start up. When I try to run it, I get back my shell prompt immediately without anything happening.

[Trev]

prism2.device shouldn't be replying to more than one request per packet. If it is, it's yet another inconsistency in the way folks develop SANA-II device drivers. (This is why companies like Apple and Microsoft provide test cases, test tools, code verification tools, and code validation services. Alas, Amiga has gone to a "better place," so we get what we get.)

Are you pinging from Amiga to Amiga? Most newer implementations of ping (the non-Microsoft ones) send a single request and display " is alive" or something similar. Microsoft sends four requests on a non-configurable interval of 1 second. The only ICMP-related message you'll receive from uIP at the moment is "icmp: not icmp echo." if an ICMP packet with a type other than echo request is received. Anyhow, which ping are you using?

Combining those previous two paragraphs, we may be able to infer that uIP is reponding nine times (BUFFERS - 1) to the first echo request (thanks to prism2.device?), and for whatever reason--probably queueing of the responses by the underlying stack--your ping client interprets those replies as bad responses (wrong sequence number) to the next eight requests. Just guessing, don't know for sure. If that's the case, I'd say the client side is broken, as it should just discard those extra packets. Regardless, we'll sort it out. Try BUFFERS 2 (the old behavior) and let me know if it still happens.

I started linking with vbcc's auto library during testing. That's probably what's preventing the application from starting on OS 2.1. I'll pull it out, as I'm not using it at the moment. I should probably start loading libraries by hand, but it's nice when the compiler and libraries do it for you. :-)

Now that WinUAE supports a SANA device, I can start testing on various combinations of Kickstart and Workbench.

Trev

[Trev]

Toni mentioned that Genesis calls S2_ONLINE before calling S2_CONFIGINTERFACE. That got me thinking once again about how device driver developers interpret the SANA-II specification. Some drivers may initialize their hardware in S2_ONLINE and some may initialize their hardware in S2_CONFIGINTERFACE. e.g.:

weird.device:

1. S2_ONLINE
2. S2_CONFIGINTERFACE
3. S2_ONLINE (?)

sane.device

1. S2_CONFIGINTERFACE
2. S2_ONLINE

This sticky point is brought to light in Sana2Device.doc/S2_ONLINE:

This command is responsible for putting the network interface hardware back into a known state (as close as possible to the state before S2_OFFLINE) and resets the unit global and special statistics.

and S2_OFFLINE:

While the interface is offline, all read, writes and any other command that touches interface hardware will be rejected with ios2_Error set to S2ERR_OUTOFSERVICE

If you interpret S2_CONFIGINTERFACE as "any other command," that would explain the problems with uIP failing to configure and bring online certain devices. I follow the semantics as described for the sane device.

Thoughts? norway.device appears to be of the weird variety, and I should have some more information on that device in particular soon.

EDIT: I should be able to just check the S2_CONFIGINTEFACE result for an S2ERR_OUTOFSERVICE value, and call S2_ONLINE if necessary. Hmmm. But that may break other devices. Maybe this should be a command-line switch? I'll call it WEIRD and leave it off by default.

Trev

[Trev]

I didn't add a WEIRD option, but I did add an extra little bit of logic to try S2_ONLINE if S2_CONFIGINTERFACE fails with S2ERR_OUTOFSERVICE or S2ERR_BAD_STATE.

If you had problems with uIP configuring or bringing a device online, please try it out and let me know what happens:

http://www.spookysoftware.com/uip.lha

This could make ariadne_ii.device, ioblixether.device, and norway.device work properly. Latest compatibility notes:


DEVICE                VERSION   WORKS

ariadne_ii.device     43.12     no
cnet.device           1.9       yes
fastethernet.device             yes
hydra.device                    yes
ioblixether.device    37.14     no
norway.device                   yes [1]
openpci_8139.device   1.2b4     yes
prism2.device                   yes
uaenet.device         1.4.5.b14 yes [2]

1. Must be configured by Miami first.
2. Does not see broadcasts from host environment (Windows).


If results still aren't consistent, we can sort out the per-device requirements with sanautil (or I can write a little test application) and perhaps put together a little compatiblity file (uip.compat or something) that tells the init routine how to configure a particular device based on name and perhaps version.

@Colani1200

I also removed the link to auto. Give it a try.

Trev

[mboehmer_e3b]

DEVICE             VERSION          WORKS
norway.device      1.8 (25.05.03)   yes

The new version works nicely with norway.device. No other config is needed anymore, I just start the uip and everything works fine.
DHCP is also working nicely with my router here.

BTW, you don't look as default into devs:network/ for the device files?

No speed tests done yet, but IBrowse (demo version) complains on not finding a "bsdsocket.library or compatible networking interface".

Anyhow, as this little bug is fixed now I'm quite confident for the future.

Thanks for your efforts! Go ahead with this amazing little stack :-)

Michael

[Piru]

No speed tests done yet, but IBrowse (demo version) complains on not finding a "bsdsocket.library or compatible networking interface".

Uhm, this thing doesn't implement bsdsocket.library (it's a linklib tcp/ip stack), so that is no bug.

[Colani1200]

Trev wrote:
Anyhow, which ping are you using?

I was pinging from FreeBSD 6.2, I suppose the client side is pretty healthy ;-)

Try BUFFERS 2 (the old behavior) and let me know if it still happens.

I'll do so, just a little patience...

I started linking with vbcc's auto library during testing. That's probably what's preventing the application from starting on OS 2.1. I'll pull it out, as I'm not using it at the moment.

Maybe the arguments are not read correctly on the 600? I double checked them, so I am quite sure that there was no typo, but uIP quits without feedback when your arguments are incorrect, so you never know... Maybe you can build in some context help?
EDIT: Didn' check your latest version on the 600 yet, I hope I can try it tonight...

[Trev]

@mboehmer_e3b

Awesome. We'll call that done. Hopefully, the workaround will fix other non-working devices as well.

You are correct. I don't search devs:networks/ or devs: for the device driver. I suppose I should add that, but it means some calls to string mangling code in the uIP driver. Extra fluff ;-) like that will hurt the stack size-wise when I start optimizing.

@Piru

Not yet, anyway. ;-) Although I'd rather turn lwIP into a bsdsocket.library implementation. It's possible with both, though.

@Colani1200

I have a feeling the problem is the device, but you never know. I'll look at the prism2 code. If there's something I'm doing incorrectly for that device, I'll need to implement a fix in a way that doesn't break other devices.

The new build dumps "required argument missing" when you pass a bad command-line. The CLI is probably the most unuserfriendly part of Amiga OS.

When auto fails to load a required library, it exits without an error message. The startup code should do the same if it fails to load dos.library. Per the vbcc documentation, they don't check for minimum versions unless you specifically ask them to with variable overrides in code. Need to double-check that. It would be nice to have the link library and startup source. :-(

Trev

[Colani1200]

Trev wrote:

I have a feeling the problem is the device, but you never know. I'll look at the prism2 code. If there's something I'm doing incorrectly for that device, I'll need to implement a fix in a way that doesn't break other devices.

I suggest you don't investigate on this any further at the moment. The problem somehow seems to be also related to my network. I have the same effect when I ping other hosts with -l 572.

Good news: The latest version runs on the A600 again :-) I have ping times of ~120 ms with cnet.device. But buffer settings don't seem to have much effect and dropped packets are still there. Maybe you want to have a look at it yourself? You can reach my A600 here: [EDIT: Link removed, offline now]

[Trev]

:-) Are you behind a firewall and using port mapping and packet forwarding for HTTP? This shouldn't be working:

Pinging wanda25.dyndns.org [82.207.247.250] with 1464 bytes of data:

Reply from 82.207.247.250: bytes=1464 time=228ms TTL=53
Reply from 82.207.247.250: bytes=1464 time=225ms TTL=53
Reply from 82.207.247.250: bytes=1464 time=221ms TTL=53
Reply from 82.207.247.250: bytes=1464 time=244ms TTL=53

Ping statistics for 82.207.247.250:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 221ms, Maximum = 244ms, Average = 229ms

Trev