Bug in AmigaOS 3.9 ram-handler ?

[Noster]

Is there anybody who can help ?

I'm currently developing a small shared runtime-library for the AmigaOS used to
access XBASE-alike databases. The library is nearly completed and I think I will
release it in the next weeks (if a few more tests have been performed).
It will be a library accessable by any Amiga-program like the system libraries
and allows shared access to databases and the according indexes.

Now I've encountered a big problem:
I've tested the library using an Amiga 3000T with AmigaOS 3.9, BoingBag 2 (I've
also tested BoingBag 1). The computer is equiped with a
CyberstormPPC/060-50/604e-200 with 138MB ram overall and an UW-SCSI drive at the
SCSI-controller of the Cyberstorm formated using FFS from OS 3.9.
A second testsystem is an A3000T with AmigaOS 3.1. This computer is equiped with
an A3640/25MHz with 18 MB ram and an (fast)SCSI-drive at the build-in
SCSI-controller, formated using FFS from OS 3.1.
The third system is an A2000 with AmigaOS 2.04, equiped with an A2630/25MHz with
0MB 32-bit RAM :-(, 1MB Chip and 4 MB 16-bit FastRam and an 30MB(!)
RLL-format(!) HD at an Omti-Controller accessed via A.L.F. 2, formated using FFS
from AmigaOS 2.04.

The library works fine on every of these Amigas, using either datatables stored
on the HD or in the ram-disk, except for the A3000T using AmigaOS 3.9 ! At this
Amiga, the ram-disk dies, if it encounters a lock-collision:

If a record is shared locked (REC_SHARED) using the AmigaDos function
"LockRecord()" and I try to lock this record exclusive (REC_EXCLUSIVE) from
another application running in another shell, this should lead to the
Dos-failure ERROR_LOCK_TIMEOUT (as it does at the other Amigas), but instead it
results in a meeting with the Guru 8000000B, a dead-end alert.

This is not a hardware-dependend error, it works if I boot the Amiga with OS
3.1 (shut off the computer, wait 10 seconds, start it again, press both
mousebuttons and select the partition with the old OS 3.1 installation as
bootdevice -- fine that I've not killed it -- and start the test-application
under AmigaOS 3.1).

Is there anybody out there, who could help me with my problem ? I won't like to
say "Here is the new database.library, enjoy it, but don't use with datatables
in the ram-disk." :-(
I think there is a problem with the OS 3.9 ram-handler, there should be a way to
say the SetPatch function, which patches the OS 3.1 libraries on system startup
with the new OS 3.9 functions, to exclude the patching of the ram-handler, so
the OS 3.1 ram-handler is used instead. The according option is named
"SKIPROMMODULES" but which name has the module to be skipped ?

I've also started a support-request by "Haage + Partner", the companie that
developed AmigaOS 3.9, but I don't have much hope they answer very fast.

      Noster

DON'T PANIC
-- Douglas Adams, Hitch Hiker's Guide to the Galaxis

[PiR]

Hi

I don't think they spoiled something working properly. However some things changed, so some things may appear, although not showing up earlier.

My first bet is running out of stack. Stack requirements for functions may change from one OS to another, and I guess the newer OS, the higher requirement...

The second choice would be calling function from Task (even not directly), where you must have complete Process structure.

There are some tools that can help you - most obviously with stack, but this looks like heavy tracing job for you.

Good luck

[Noster]

Thanks for trying

> My first bet is running out of stack.
> Stack requirements for functions may change from one OS to another, and I guess the newer OS, the higher requirement...

No, the application runs in the same environment (shell), one time accessing the disk (without any problem), the other time accessing the ram-disk, with the Guru as result.
The test uses the same code, the same variables, all is the same, so the stack consumtion is also the same, this cannot be the problem.

> The second choice would be calling function from Task (even not directly),
> where you must have complete Process structure.

Dito

But I've found a topic in the forum of Amiga-News from the summer 2002, where a user also has Guru with the ram-handler.
There a helpful person told the name of the module required for "SetPatch". The line in startup-sequence should be "SetPatch SKIPROMMODULES ram-handler quiet"
I will test it now and post the result.

Noster.

[cha05e90]

Hi!

Skipping the ram-handler module is one possible solution, but not very elegant...:-), especially when other people want to try your library WITH the ram-handler.
Did you know that there are some semi-official patches for OS3.9-BB2? A lot of people do not know them, because they are not officially released by Amiga Inc. or Haage & Partner, BUT they are from the guy who actually wrote those library for OS3.9 (his name is Thomas Richter/THOR). There were at least two patches for OS3.9 stuff (after! BB2) released from him to aminet:

util/boot/patchram
util/boot/shellupdate

Maybe the new version of the ram-handler can cure your problem.

Regards,
Frank

[prowler]

Hrmm, I wonder if this patchram from Aminet will help me with my ram-lib guru problem under os3.9 when I boot my cyberstorm mk-i up with RAM on the card (in a 3000T) Only one way to find out I suppose.

[Noster]

Hi Frank,

> Skipping the ram-handler module is one possible solution,
> but not very elegant..., especially when other people want
> to try your library WITH the ram-handler.

The library works, but you have to place the files of the database on the harddisk, NOT in the ram-disk (if you want to use the OS 3.9 Ram-Handler).
 
> Did you know that there are some semi-official patches for OS3.9-BB2?
> ...
> There were at least two patches for OS3.9 stuff ...
>
> util/boot/patchram
> util/boot/shellupdate

Thanks for the tip, the "shellupdate" was known by me and already installed, but not the Ram-Handler patch.

> Maybe the new version of the ram-handler can cure your problem.

Sick...No it doesn't, as mentioned in the according Read.Me, it only changes the displayed size of the ram-disk, nothing else.
But this way I've got the eMail-address of Thomas Richter, and send a description of my problem to him. If he has written the Ram-Handler, he may be able to help me (and the rest of the Amiga-world, if it's a real bug).

Now lets wait, what the future brings...by the way, Haage+Partner have answered (very fast, wow) but they are only the publisher of AmigaOS 3.9, they don't know programming-internals, but they give me the tip with the Amiga-News forum.
If I skip the ram-handler update, my library works perfekt, one more indicator of a bug in that handler-code.

Noster

[ncafferkey]

I don't think Thomas Richter wrote the ram-handler.

I wrote a test program just now, and got a crash too. However, I found that if the second attempt to lock the record (exclusively) is done with REC_EXCLUSIVE_IMMED, it doesn't crash. Maybe this could be a useful work-around for you.

[lempkee]

hi , ok this problem is very known and has been fixed by many allready.

if you run MCP then use that and select the fix there.

if you dont use MCP , well then use MCPRAMLIBPATCH (put it in C: and start it after setpatch .

if you need this file , email me or PM me and remeber to leave your email so i know where to send it..

cheers

[Noster]

Hi

> However, I found that if the second attempt to lock the record
> (exclusively) is done with REC_EXCLUSIVE_IMMED, it doesn't crash.
> Maybe this could be a useful work-around for you.

Maybe, I will test it immediately...

I've also written a test-tool this day, that tests all AmigaDos functions that are filesystem dependend and used by my library and encounterd another problem :-(
The function ChangeMode() doesn't works correct in conjunction with the ram-disk:
If you open a file twice (with shared access, MODE_OLDFILE) and try to change the mode of one of these filehandles to exclusive access using ChangeMode() this should be fail and set the IoErr() ERROR_OBJECT_IN_USE -- as it does if the file is located on a FFS-formated HD-partion. But if the file is located in ram-disk, the function succeeds ??! How could I gain exclusive access, if another filehandle is still valid ?

What a trash...:-|

   Noster

[Noster]

Hi,

I've tested to use the REC_SHARED_IMMED and the REC_EXCLUSIVE_IMMED flags and they work. Ok that's fine so far, but life would be some much easier, if I could wait for a timeout and let the Ram-Handler awake my process, if the timeout is elapsed or the lock is free...
It's hard to do this on my own, this would lead to something like "polling": looking every 1/10 of a second if the lock is free now...that is not very multitasking-friendly :-(

The MCPRAMLIBPATCH is also not solving this problem, it patches the displayed ram used by the ram-disk, instead of displaying "100% used, 0KB free" it shows the real percentage used and the systemwide available free memory, but who cares about what is displayed in the border of that window ?

Thomas Richter has also answered, but he has only made the little patch available at Aminet that does the same as the MCPRAMLIBPATCH. His own Amiga is died 2 years ago and he doesn't do anything for this system any more. But he has given me another eMail-address of the one, he thinks has worked on the Ram-Handler, but the address is over 2 years old. Lets see if someone is answering...

Still searching for a solution, Noster

[Piru]

The MCPRAMLIBPATCH is also not solving this problem, it patches the displayed ram used by the ram-disk, instead of displaying "100% used, 0KB free" it shows the real percentage used and the systemwide available free memory, but who cares about what is displayed in the border of that window ?

Wrong, it has nothing to do with ram disk.

It patch ramlib (module responsible for loading/unloading libraries, devices and resources) task stack to 8KB instead of the original 2KB.

This cures various crashes with buggy libraries, for example Warp3D, that would otherwise overflow the ramlib stack in LIB_INIT.

[Piru]

hi , ok this problem is very known and has been fixed by many allready.

First time I hear about it. This sounds like ACTION_LOCK_RECORD bug in ram-handler.

if you run MCP then use that and select the fix there.

MCP has no fix for this bug.

if you dont use MCP , well then use MCPRAMLIBPATCH (put it in C: and start it after setpatch.

MCPRAMBLIPATCH doesn't fix this problem. It only fixes the ramlib task stacksize.

[ncafferkey]

I think lempkee was responding to prowler's post, which had nothing to do with the main topic of this thread.

I've tested to use the REC_SHARED_IMMED and the REC_EXCLUSIVE_IMMED flags and they work. Ok that's fine so far, but life would be some much easier, if I could wait for a timeout and let the Ram-Handler awake my process, if the timeout is elapsed or the lock is free...
It's hard to do this on my own, this would lead to something like "polling": looking every 1/10 of a second if the lock is free now...that is not very multitasking-friendly

Even if the timeout modes worked, I think the maximum timeout you can specify is 50 ticks, so you'd still have to poll every second.

I can't actually see why you need to lock records twice with different handles. Are you sure there isn't a better way to achieve what you're trying to do?

[Noster]

Hi

> Even if the timeout modes worked, I think the maximum timeout you can specify is 50 ticks...

No, there is no upper limit...Ok, 2147483647 Ticks :-)

> I can't actually see why you need to lock records twice with different handles.

Ok, this might not be required for the ram-disk, but how should the library detect, which kind of filesystem is used for the files and should it do an own locking algorithm for every kind of filesystem ?
The trick will be that the files should also be placeable on a network filesystem and several Amigas could share the files simultaneous. So I need to lock the records while they are in access by one system to avoid conflicts.

Ok, you may say "use MySQL" for these requirements, but why should I use such an overhead of an SQL database-management-system (DBMS), if the database just contains a few tables with 50000 records or less ? For these little databases something like Clipper is the better choise. The other benefit for using recordlocking instead of a DBMS is, that I don't need to write a DBMS for the server, resp. doesn't need to run one on the server. So I could use any kind of server: another Amiga, a Linux system or a Windoof server, they just need to store the files, nothing else.

But with these problems -- I've encountered -- I think I know why noone has released a working database.library till now.

   Noster

[ncafferkey]

Does the problem with the RAM-handler only arise when more than one program is using your library? If so, is this not acceptable given that that the bug isn't in your code. People will still be able to use it with the RAM-handler most of the time.