Bug in AmigaOS 3.9 ram-handler ?

[Noster]

Is there anybody who can help ?

I'm currently developing a small shared runtime-library for the AmigaOS used to
access XBASE-alike databases. The library is nearly completed and I think I will
release it in the next weeks (if a few more tests have been performed).
It will be a library accessable by any Amiga-program like the system libraries
and allows shared access to databases and the according indexes.

Now I've encountered a big problem:
I've tested the library using an Amiga 3000T with AmigaOS 3.9, BoingBag 2 (I've
also tested BoingBag 1). The computer is equiped with a
CyberstormPPC/060-50/604e-200 with 138MB ram overall and an UW-SCSI drive at the
SCSI-controller of the Cyberstorm formated using FFS from OS 3.9.
A second testsystem is an A3000T with AmigaOS 3.1. This computer is equiped with
an A3640/25MHz with 18 MB ram and an (fast)SCSI-drive at the build-in
SCSI-controller, formated using FFS from OS 3.1.
The third system is an A2000 with AmigaOS 2.04, equiped with an A2630/25MHz with
0MB 32-bit RAM :-(, 1MB Chip and 4 MB 16-bit FastRam and an 30MB(!)
RLL-format(!) HD at an Omti-Controller accessed via A.L.F. 2, formated using FFS
from AmigaOS 2.04.

The library works fine on every of these Amigas, using either datatables stored
on the HD or in the ram-disk, except for the A3000T using AmigaOS 3.9 ! At this
Amiga, the ram-disk dies, if it encounters a lock-collision:

If a record is shared locked (REC_SHARED) using the AmigaDos function
"LockRecord()" and I try to lock this record exclusive (REC_EXCLUSIVE) from
another application running in another shell, this should lead to the
Dos-failure ERROR_LOCK_TIMEOUT (as it does at the other Amigas), but instead it
results in a meeting with the Guru 8000000B, a dead-end alert.

This is not a hardware-dependend error, it works if I boot the Amiga with OS
3.1 (shut off the computer, wait 10 seconds, start it again, press both
mousebuttons and select the partition with the old OS 3.1 installation as
bootdevice -- fine that I've not killed it -- and start the test-application
under AmigaOS 3.1).

Is there anybody out there, who could help me with my problem ? I won't like to
say "Here is the new database.library, enjoy it, but don't use with datatables
in the ram-disk." :-(
I think there is a problem with the OS 3.9 ram-handler, there should be a way to
say the SetPatch function, which patches the OS 3.1 libraries on system startup
with the new OS 3.9 functions, to exclude the patching of the ram-handler, so
the OS 3.1 ram-handler is used instead. The according option is named
"SKIPROMMODULES" but which name has the module to be skipped ?

I've also started a support-request by "Haage + Partner", the companie that
developed AmigaOS 3.9, but I don't have much hope they answer very fast.

      Noster

DON'T PANIC
-- Douglas Adams, Hitch Hiker's Guide to the Galaxis

[Noster]

Thanks for trying

> My first bet is running out of stack.
> Stack requirements for functions may change from one OS to another, and I guess the newer OS, the higher requirement...

No, the application runs in the same environment (shell), one time accessing the disk (without any problem), the other time accessing the ram-disk, with the Guru as result.
The test uses the same code, the same variables, all is the same, so the stack consumtion is also the same, this cannot be the problem.

> The second choice would be calling function from Task (even not directly),
> where you must have complete Process structure.

Dito

But I've found a topic in the forum of Amiga-News from the summer 2002, where a user also has Guru with the ram-handler.
There a helpful person told the name of the module required for "SetPatch". The line in startup-sequence should be "SetPatch SKIPROMMODULES ram-handler quiet"
I will test it now and post the result.

Noster.

[Noster]

Hi Frank,

> Skipping the ram-handler module is one possible solution,
> but not very elegant..., especially when other people want
> to try your library WITH the ram-handler.

The library works, but you have to place the files of the database on the harddisk, NOT in the ram-disk (if you want to use the OS 3.9 Ram-Handler).
 
> Did you know that there are some semi-official patches for OS3.9-BB2?
> ...
> There were at least two patches for OS3.9 stuff ...
>
> util/boot/patchram
> util/boot/shellupdate

Thanks for the tip, the "shellupdate" was known by me and already installed, but not the Ram-Handler patch.

> Maybe the new version of the ram-handler can cure your problem.

Sick...No it doesn't, as mentioned in the according Read.Me, it only changes the displayed size of the ram-disk, nothing else.
But this way I've got the eMail-address of Thomas Richter, and send a description of my problem to him. If he has written the Ram-Handler, he may be able to help me (and the rest of the Amiga-world, if it's a real bug).

Now lets wait, what the future brings...by the way, Haage+Partner have answered (very fast, wow) but they are only the publisher of AmigaOS 3.9, they don't know programming-internals, but they give me the tip with the Amiga-News forum.
If I skip the ram-handler update, my library works perfekt, one more indicator of a bug in that handler-code.

Noster

[Noster]

Hi

> However, I found that if the second attempt to lock the record
> (exclusively) is done with REC_EXCLUSIVE_IMMED, it doesn't crash.
> Maybe this could be a useful work-around for you.

Maybe, I will test it immediately...

I've also written a test-tool this day, that tests all AmigaDos functions that are filesystem dependend and used by my library and encounterd another problem :-(
The function ChangeMode() doesn't works correct in conjunction with the ram-disk:
If you open a file twice (with shared access, MODE_OLDFILE) and try to change the mode of one of these filehandles to exclusive access using ChangeMode() this should be fail and set the IoErr() ERROR_OBJECT_IN_USE -- as it does if the file is located on a FFS-formated HD-partion. But if the file is located in ram-disk, the function succeeds ??! How could I gain exclusive access, if another filehandle is still valid ?

What a trash...:-|

   Noster

[Noster]

Hi,

I've tested to use the REC_SHARED_IMMED and the REC_EXCLUSIVE_IMMED flags and they work. Ok that's fine so far, but life would be some much easier, if I could wait for a timeout and let the Ram-Handler awake my process, if the timeout is elapsed or the lock is free...
It's hard to do this on my own, this would lead to something like "polling": looking every 1/10 of a second if the lock is free now...that is not very multitasking-friendly :-(

The MCPRAMLIBPATCH is also not solving this problem, it patches the displayed ram used by the ram-disk, instead of displaying "100% used, 0KB free" it shows the real percentage used and the systemwide available free memory, but who cares about what is displayed in the border of that window ?

Thomas Richter has also answered, but he has only made the little patch available at Aminet that does the same as the MCPRAMLIBPATCH. His own Amiga is died 2 years ago and he doesn't do anything for this system any more. But he has given me another eMail-address of the one, he thinks has worked on the Ram-Handler, but the address is over 2 years old. Lets see if someone is answering...

Still searching for a solution, Noster

[Noster]

Hi

> Even if the timeout modes worked, I think the maximum timeout you can specify is 50 ticks...

No, there is no upper limit...Ok, 2147483647 Ticks :-)

> I can't actually see why you need to lock records twice with different handles.

Ok, this might not be required for the ram-disk, but how should the library detect, which kind of filesystem is used for the files and should it do an own locking algorithm for every kind of filesystem ?
The trick will be that the files should also be placeable on a network filesystem and several Amigas could share the files simultaneous. So I need to lock the records while they are in access by one system to avoid conflicts.

Ok, you may say "use MySQL" for these requirements, but why should I use such an overhead of an SQL database-management-system (DBMS), if the database just contains a few tables with 50000 records or less ? For these little databases something like Clipper is the better choise. The other benefit for using recordlocking instead of a DBMS is, that I don't need to write a DBMS for the server, resp. doesn't need to run one on the server. So I could use any kind of server: another Amiga, a Linux system or a Windoof server, they just need to store the files, nothing else.

But with these problems -- I've encountered -- I think I know why noone has released a working database.library till now.

   Noster

[Noster]

Hi

(sorry for the late answer, I've been offline for two days)

> ncafferkey wrote:
> Does the problem with the RAM-handler only arise when more than one program is using your library?

Yes that's true, even if more than one program accesses the same files in the ram-disk via my library, the ram-handler will not die, unless a lock-fault appears, i.e. one program still holds a recordlock, when another is trying to obtain a lock. (I've constructed this situation in my test-program, usually it is not a very frequent event).

But nevertheless this bug is annoying, it should not be. It is so easy to create a test for this condition, it should have been testet, found, and removed when the ram-handler was written BEFORE it was released.

Noster

[Noster]

Hi

> I wonder if the Hyperion team knows about this problem?

I don't know, all email addresses I've tried until now to find someone who has access to the source, to be able to remove this bug, have either not responded or was the wrong person, don't able to help :-(
But I haven't tried to contact Hyperion, I'm looking for a solution for OS 3.9.

> Is there any chance this ram-handler bug will be part of OS4?

If it will be, I will find it :-)
I hope that in that case the bug will be removed by the first available BoingBag, that the OS 4.0 will not be a dead child like OS 3.9.

> Noster, How about an update on your runtime-library?

If my database.library is completed (currently I'm writing the documentation, examplecode and removing errors found by these examples) and OS 4.0 is available for my A3000T with CyberstormPPC, I will try to create a OS 4.0 PPC version, of course.

Noster