uIP/lwIP for Amiga OS

[Trev]

Multiple receive buffers have been implemented and a command-line option (BUFFERS) has been added. The minimum number of buffers is two--one for IP and one for ARP. Currently, the first buffer is reserved for ARP and the rest are used for IP. Each buffer consumes ~600 additional bytes of memory. Three or more buffers provide a decent level of interactivity.

The device I'm using (openpci_8139.device) sends "orphaned" frames to all pending S2_READORPHAN requests. I'm not sure if all devices do that, but I suppose it makes sense. Device driver authors have no way to determine the intent of a task or process when it queues multiple S2_READORPHAN requests. Anyhow, I was receiving multiple copies of frames after implementing mulitple receive requests. That lead to the split between ARP and IP requests. Reads and writes are still raw (the entire frame minus the ethernet trailer is copied to the buffer), but reads will only catch ARP and IP frame types.

EDIT: You know, a device also has no way of knowing how a process or task is going to use multiple ARP or IP requests. As devices are required to service requests for all callers as of S2R3, device driver authors would have to make a conscious decision to send one frame per CMD_READ request once per task or process and not the same frame across multiple CMD_READ requets. Not sure why the same decision wasn't made for S2_READORPHAN requests. More digging to do, I guess.

The PNG image in files.shtml failed to transmit because of a bug in the Perl script that converts content into C. The script was using read(), which wraps stdio. On Windows, CRLF sequences are converted to LF. Not a good thing for binary files. Replaced read() with sysread() to fix that.

Still have checksums to implement. I also have a few ideas about why certain devices aren't initialized properly.

[Trev]

A 680x0 specific checksum routine has been shamelessly stolen from RTEMS (uses a BSD stack) and modified for vbcc. Oh, the joys of open source software. Cycle-wise, it's about 250% faster than uIP's default routine (but only a pinch faster than the BSD optimized C routine). Being BSD-derived, the routines are very similar to AmiTCP's. I'm steering clear of AmiTCP code, however, since it's GPL. I'd rather stick with a BSD-style license that doesn't place any restrictions on use or distribution (short of copyright notices).

Toni Wilen has added a SANA-II device (uaenet.device) to the latest builds of WinUAE. Very cool, but it currently requires TAP-Win32. I'd rather see an implementation that doesn't require a bridged virtual network adapter. Microsoft Virtual PC does it (of course), and so does nmap. Regardless, cheers to Toni for all the hard work.

Still confused as to why some adapters configure and go online and some don't. For those that go online after being configured by another stack, e.g. Miami, the problem is obviously with my initialization code.

If anyone wants to take a stab at using sanautil (available on Aminet) to configure and bring a card online, please let me know the results. I'm particularly intersted in the output of "sanautil device status" before and after the card is configured and before and after the card is brought online.

Here's the compatibility list as it stands today:

ariadne_ii.device,43.12,no
cnet.device,1.9,yes
hydra.device,?,yes
ioblixether.device,37.14,no
norway.device,?,yes*
openpci_8139.device,1.2b4,yes
prism2.device,?,yes
uaenet.device,1.0 (1.4.5b13),yes**

* after configuration by separate stack
** acquires DHCP address; hangs on first ICMP echo reply

Trev

[Trev]

My checksum routine wasn't working 100% of the time, so I had to put together a nice set of test cases. Anyhow, it should be good to go now. Here's the current set of command-line options:

Code: [Select]


/*
 * Command-line options.
 *
 * Default: DEVICE Devs:Networks/openpci_8139.device UNIT 0 CONFIG DYNAMIC
 *
 * DEVICE
 *   Fully-qualified path to device to open.
 *
 *   Example: DEVICE Devs:Networks/openpci_8139.device
 *
 * UNIT
 *   Unit number to open.
 *
 *   Example: UNIT 0
 *
 * BUFFERS
 *   Number of buffers to allocate.
 *
 *   Example: BUFFERS 10
 *
 * CONFIG
 *   IP address configuration. Valid values are DYNAMIC and STATIC. If
 *   STATIC, the ADDR, MASK, and DEST options are required.
 *
 *   Example: CONFIG DYNAMIC
 *   Example: CONFIG STATIC
 *
 * ADDR
 *   IP address. Required if CONFIG STATIC is specified.
 *
 *   Example: ADDR 192.168.0.100
 *
 * MASK
 *   IP subnet mask. Required if CONFIG STATIC is specified.
 *
 *   Example: MASK 255.255.255.0
 *
 * DEST
 *   IP default gateway. Required if CONFIG STATIC is specified.
 *
 *   Example: DEST 192.168.0.1
 */


And a link to the latest revision:

http://www.spookysoftware.com/uip.lha

I still expect folks to have problems configuring and bringing online certain adapters. If anyone gets a chance to run sanautil as requested above or knows the ins and outs of configuration for a particular device, please let me know.

Apart from that, this revision should be more responsive--both uIP and the OS itself--than the last. I'm interested in ping response times for various Amiga configurations using a full-sized frame, e.g. ping -l 572 n.n.n.n.

For best performance, use three or more buffers (the minimum is two). After that, it's a memory/performance tradeoff. There's also a point where increasing the buffers won't help, as uIP only processes pending outbound frames twice per second (configurable in code). That's why the shtml pages are so slow--the data is sent periodically.

EDIT: Oops. Forgot to put a buffer size check back in. Binary updated. Note, the URL has changed.

Trev

[Trev]

Another little revision. I modified the DHCP client to randomize the transaction ID field per process and use the client indentifier option. Assuming your ethernet device supports it, this should allow you to run multiple instances of uIP on the same interface and still allow use of DHCP.

This would be required, for example, if you wanted to embed uIP in multiple applications and run them simultaneously. Each application would have its own IP address. This won't work if your device only sends an incoming IP packet to the next pending IO request. If it sends the packet to the next pending IO request per process, then you're good to go.

Caveat: I haven't implemented DHCPRELEASE, so keep an eye on your DHCP scope(s).

http://www.spookysoftware.com/uip.lha

EDIT: Also, Toni has switched from TAP-Win32 to WinPCAP for the new uaenet.device. Looks even more promising now, although there are some new issues to resolve. We all know it, but WinUAE is one awesome piece of software. Everyone involved in UAE's development over the last 13 years (has it really been that long? yikes) really deserves our gratitude.

Trev

[Trev]

prism2.device shouldn't be replying to more than one request per packet. If it is, it's yet another inconsistency in the way folks develop SANA-II device drivers. (This is why companies like Apple and Microsoft provide test cases, test tools, code verification tools, and code validation services. Alas, Amiga has gone to a "better place," so we get what we get.)

Are you pinging from Amiga to Amiga? Most newer implementations of ping (the non-Microsoft ones) send a single request and display " is alive" or something similar. Microsoft sends four requests on a non-configurable interval of 1 second. The only ICMP-related message you'll receive from uIP at the moment is "icmp: not icmp echo." if an ICMP packet with a type other than echo request is received. Anyhow, which ping are you using?

Combining those previous two paragraphs, we may be able to infer that uIP is reponding nine times (BUFFERS - 1) to the first echo request (thanks to prism2.device?), and for whatever reason--probably queueing of the responses by the underlying stack--your ping client interprets those replies as bad responses (wrong sequence number) to the next eight requests. Just guessing, don't know for sure. If that's the case, I'd say the client side is broken, as it should just discard those extra packets. Regardless, we'll sort it out. Try BUFFERS 2 (the old behavior) and let me know if it still happens.

I started linking with vbcc's auto library during testing. That's probably what's preventing the application from starting on OS 2.1. I'll pull it out, as I'm not using it at the moment. I should probably start loading libraries by hand, but it's nice when the compiler and libraries do it for you. :-)

Now that WinUAE supports a SANA device, I can start testing on various combinations of Kickstart and Workbench.

Trev

[Trev]

Toni mentioned that Genesis calls S2_ONLINE before calling S2_CONFIGINTERFACE. That got me thinking once again about how device driver developers interpret the SANA-II specification. Some drivers may initialize their hardware in S2_ONLINE and some may initialize their hardware in S2_CONFIGINTERFACE. e.g.:

weird.device:

1. S2_ONLINE
2. S2_CONFIGINTERFACE
3. S2_ONLINE (?)

sane.device

1. S2_CONFIGINTERFACE
2. S2_ONLINE

This sticky point is brought to light in Sana2Device.doc/S2_ONLINE:

This command is responsible for putting the network interface hardware back into a known state (as close as possible to the state before S2_OFFLINE) and resets the unit global and special statistics.

and S2_OFFLINE:

While the interface is offline, all read, writes and any other command that touches interface hardware will be rejected with ios2_Error set to S2ERR_OUTOFSERVICE

If you interpret S2_CONFIGINTERFACE as "any other command," that would explain the problems with uIP failing to configure and bring online certain devices. I follow the semantics as described for the sane device.

Thoughts? norway.device appears to be of the weird variety, and I should have some more information on that device in particular soon.

EDIT: I should be able to just check the S2_CONFIGINTEFACE result for an S2ERR_OUTOFSERVICE value, and call S2_ONLINE if necessary. Hmmm. But that may break other devices. Maybe this should be a command-line switch? I'll call it WEIRD and leave it off by default.

Trev

[Trev]

I didn't add a WEIRD option, but I did add an extra little bit of logic to try S2_ONLINE if S2_CONFIGINTERFACE fails with S2ERR_OUTOFSERVICE or S2ERR_BAD_STATE.

If you had problems with uIP configuring or bringing a device online, please try it out and let me know what happens:

http://www.spookysoftware.com/uip.lha

This could make ariadne_ii.device, ioblixether.device, and norway.device work properly. Latest compatibility notes:


DEVICE                VERSION   WORKS

ariadne_ii.device     43.12     no
cnet.device           1.9       yes
fastethernet.device             yes
hydra.device                    yes
ioblixether.device    37.14     no
norway.device                   yes [1]
openpci_8139.device   1.2b4     yes
prism2.device                   yes
uaenet.device         1.4.5.b14 yes [2]

1. Must be configured by Miami first.
2. Does not see broadcasts from host environment (Windows).


If results still aren't consistent, we can sort out the per-device requirements with sanautil (or I can write a little test application) and perhaps put together a little compatiblity file (uip.compat or something) that tells the init routine how to configure a particular device based on name and perhaps version.

@Colani1200

I also removed the link to auto. Give it a try.

Trev

[Trev]

@mboehmer_e3b

Awesome. We'll call that done. Hopefully, the workaround will fix other non-working devices as well.

You are correct. I don't search devs:networks/ or devs: for the device driver. I suppose I should add that, but it means some calls to string mangling code in the uIP driver. Extra fluff ;-) like that will hurt the stack size-wise when I start optimizing.

@Piru

Not yet, anyway. ;-) Although I'd rather turn lwIP into a bsdsocket.library implementation. It's possible with both, though.

@Colani1200

I have a feeling the problem is the device, but you never know. I'll look at the prism2 code. If there's something I'm doing incorrectly for that device, I'll need to implement a fix in a way that doesn't break other devices.

The new build dumps "required argument missing" when you pass a bad command-line. The CLI is probably the most unuserfriendly part of Amiga OS.

When auto fails to load a required library, it exits without an error message. The startup code should do the same if it fails to load dos.library. Per the vbcc documentation, they don't check for minimum versions unless you specifically ask them to with variable overrides in code. Need to double-check that. It would be nice to have the link library and startup source. :-(

Trev

[Trev]

:-) Are you behind a firewall and using port mapping and packet forwarding for HTTP? This shouldn't be working:

Pinging wanda25.dyndns.org [82.207.247.250] with 1464 bytes of data:

Reply from 82.207.247.250: bytes=1464 time=228ms TTL=53
Reply from 82.207.247.250: bytes=1464 time=225ms TTL=53
Reply from 82.207.247.250: bytes=1464 time=221ms TTL=53
Reply from 82.207.247.250: bytes=1464 time=244ms TTL=53

Ping statistics for 82.207.247.250:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 221ms, Maximum = 244ms, Average = 229ms

Trev

[Trev]

I didn't see any when I looked. I'll put together a little list of events/scenarios that would result in a dropped packet counter being incremented, and we'll go from there.

Trev

[Trev]

These are the conditions that will increment the drop counters:

TCP:
invalid header
invalid header checksum
invalid checksum
fragemented packet
wrong IP address (not a broadcast)
a protocol other than TCP, UDP, or ICMP

UDP:
invalid checksum

ICMP:
a type other than ECHO

I've also put up a new version that includes support for renewing DHCP leases. The lease times will be displayed along with the rest of the configuration. When the lease is renewed, it just spits out the configuration again. If the lease is not renewed, the IP address is set to 0.0.0.0 (may need to make that 255.255.255.255--haven't tested this scenario) and the DHCP discover/request/renew process starts over.

Releasing DHCP addresses isn't strictly required (the DHCP server will clean up leases when they expire), and adding release support will require non-trivial changes to the DHCP client. I'll look at it again when I get around to the non-trivial stuff (read: bsdsocket.library). ... Actually, as I was typing that, I thought of a trivial way to implement it. I'll try it out tomorrow.

You'll find uip and uip-debug in the archive. uip-debug has a little bit of extra logging information and checksum output enabled.

http://www.spookysoftware.com/uip.lha

Trev

[Trev]

Link-local addresses aren't implemented. Implementation of that and other zero configuration networking bits isn't a high priority for me; however, link-local addresses, multicast DNS, et al are very Amiga-like things. They'd also make interaction with Windows, OS X, modern printers, consumer NAS devices, and other cool things pretty straightforward for end users (assuming the required protocols were there, too). When implemented properly, Zeroconf "just works."

Trev

[Trev]

For anyone that's interested, lwIP is coming along as well. It compiles (obviously, that's not the difficult part), but the Amiga-specific stuff isn't really working at the moment. It's a much bigger stack, closer to the BSD stack (AmiTCP) in complexity, depending on what you choose to implement.

I'll have something to share in the next week or so, I think, hopefully with full bsdsocket.library emulation. No promises. Worst case, it'll be similar in nature to the current uIP port--a proof of concept to show folks that it works.

For both stacks, I really do intend to have source code up. Seriously--I need advice on how best to optimize the Amiga bits for the smallest footprint possible.

And good news for me, my grandmother's boyfriend (yes, grandparents date, too, apparently) explained to me how a tap works and what I need to buy to drill and tap properly. I'm obviously not mechanically inclined. My Z4 tower arrived with the Z4 busboard holes off-center (apparently, there was more than one revision of the Z4? or maybe Eyetech goofed when they pre-drilled the last set), so my "pre-built" tower ended up being a DIY project anyway. I'm not much of a DIY guy, so we'll see how it goes.

Trev

[Trev]

R1 and R2 are on AmiNet (SANA.lha and sana2_v2.lha, respectively). R3 is on Developer CD V2.1, and R4 was posted on the Amiga, Inc. developer forum a number of years ago. R4 is probably also part of the OS4 SDK. The headers themselves aren't that useful. PM your email address, and I'll point you in the right direction. If you can find a copy of Developer CD V2.1 somewhere, it's a must have. If you already have the CD, look in Extras/Networking/ and DevInfo/Networking/. There are various other bits located throughout the CD. It's a bit of a mess sorting it out and putting it all together.

The best thing to do is download the source code for cnet.device, prism2.device, and other devices and look at how they're implemented. The upcoming release of WinUAE will have some interesting source code to look at as well.

Be mindful of your interpretations of S2_ONLINE and S2_CONFIGINTERFACE and be sure to test with AmiTCP 3.0, AmiTCP 4.x, Genesis, and Miami. You'll definitely encounter odd behavior. (See notes in previous posts.)

Trev

[Trev]

No lwIP yet. Got busy last week, and I proposed to my girlfriend (now my fiancee) on Monday and really haven't had time to code. :-) We're going to be apart for Christmas :-( ... so I guess that's good for lwIP. Anyhow, lwIP is multithreaded, and implementing a workable thread abstraction is a pain in the ass. Task or process? Task, I say, but that means implementing a messaging subsystem for debug output and quadruple-checking lwIP for code paths that might fall into areas not accessible to tasks. Blah.

[Trev]

Still busy, but haven't put lwIP aside. Will be out of town for a bit and then back to work.

Went with processes instead of tasks, as the core lwIP code wants functions that fall into dos.library. The stack is up and running with a loopback adapter but occassionally crashes. D'oh. Got to find out where, fix it, and add ethernet support. (It's probably just crashing on my debug output. Console I/O is wrapped with a semaphore--is that enough to make it thread safe?--but there are probably some stray printf statements somewhere causing havoc. Or it could be something else entirely. Need to do what I was talking about and implement a separate process that does nothing but accept and output debug messages.)

Trev