Hyperion announces OS 3.1 update

[Gulliver]

Hi, I have a couple of questions regarding the 3.1 update:

Will there be a CD32 workbench version support? Or will CD32 users still have to resort to hacks and custom built workbench cdroms? (freeanim+joymouse+setcdreboot+cdaudio+OSK)

And then, what about also adding CTDV workbench support with the RMTM command?

HDToolbox unfriendly brother prod_prep will be updated to support large drives?

Will you also include the quite handy but never directly included "reboot" and "findresident" commands?

Thank you for your time.

[Gulliver]

@olsen
@Thomas Richter

Thank you both for explaining the difficult situation regarding both the CDTV and the CD32.

I believe prod_prep is often forgotten, underlooked and unapreciated, and it is a very powerful tool that all amiga users should know about. Dealing with it, is certainly not easy, and could be made a little bit less difficult by including an amigaguide manual with plenty usage examples.

I personally use prod_prep for deploying my own customized system/workbench to my amigas from within a simple script. It is very small and it is a formidable tool for emergency install floppies, where space is severely restricted. This way you save space for other important components and dont need to put both format and hdtoolbox which are quite large compared to prod_prep. Furthermore, it is a bonus that you are not required to use it in an interactive way and that you can fully automate it.

An RDB library sounds like a very good idea.

Will you please replace graphicdump with a more usable snapshot tool?
It is archaic and not flexible at all. A simple screen grabber that saves to disk would suit much better, and even more now, taking into account the sad state of the printing system.

What about picture.datatype? Do you still have the v45 sources? And even a v45 rebuild without the infamous StormC compiler would be apreciated. I dont enjoy the fat binary aspect of its last incarnation in 3.9. I dont want to deal with code that I wont use. But I still want the 24 bit support.
And of course, I wont even need to mention all the datatypes that need updates. It seems a lot of work is here waiting to drive you crazy.

Regarding audio.device: do you have some plans to somehow incorporate AHI into it, so that old applications can finally become more friendly with soundcards? Afterall, unlike the RTG scenario, AHI is open source and it has become the unique and default standard for modern audio applications with no other counterpart attempting to replicate that on the Amiga.

Speaking about usefull commands, yes, my idea was to have both reboot and findresident installed. BTW, will you also include "warnifpressed". I believe you have the sources for all of these.

Thanks again.

[Gulliver]

@Thomas Richter
@olsen

Thank you for being so open and honest to discuss about the update development. Your answers are truly interesting.

Which version number will this update feature? V46? Or just whatever version number comes out from incrementing older components?

Will it feature a less broken cdfilesystem (like cachecdfs in 3.9) and will it have UDF support this time?

Do you plan on including atapiismajik (idefix97 kind of clone), from OS4 Classic, to deal with 4 channel IDE interfaces?

And please, dont waste your time updating for large drive support for the also ancient HDBackup + BRU twin brothers. I have never seen anyone use that pair, and probably for good reason. You could even replace it with former 3rd party commercial tools such as ABackup, which has been recently open sourced, entirely written in SAS/C and residing in Aminet (http://aminet.net/disk/bakup/abackup_src.lha).

[Gulliver]

@olsen
@Thomas Richter

Hi, back again. Here is a list of old bugs/abnormalities, have they been addressed?
Sorry for the long post.


utility.library (bug reported by Peter Keunecke)
===============
There is a potential risk of getting asystemcrash in some rare
cases caused by a dangerous procedure in the ROM-function
CloneTagItems(), which does not switch off the multitasking
during the copy routine. Whenever another task uses
RefreshTagItemClones() or other means to change the source
TagList while the copy is still running, it could happen, that
CloneTagItems runs out of memory and trashes the next memblock.

exec.library (bugs reported by Harry Sintonen)
============
exec/Alert() causes privilege violation trap with 68010+
CPUs if VBR is not at zero and the alert is of DEADEND type (very
annoying bug).

exec/FreePooled() not releasing memory. Empty puddles would not
get returned to the free memory list until the pool was DeletePool()ed
(minor but yet annoying bug).

exec/ReleaseSemaphore() trashing d0/d1/a0 when calling release
without obtain (minor bug, can only occur with misbehaving exec/Alert()
patch).

Missing BOOL return code workaround to exec/CheckIO() routine. The new
code should try to figure out if the caller is going to interpret the result
wrong, and if this is the case, will fix the return code (minor bug).

exec/OpenLibrary() not passing open-version in d0 for LIB_OPEN,
unlike the example sources suggest. (minor bug)

SAD/TURN_ON_SINGLE returning crap as old trace vector address
and SAD/TURN_OFF_SINGLE sending 4 bytes of crap after command DONE.
(minor bugs, as no-one really use SAD)

Return code workaround for old amiga.lib (upto version 40.15)
CreateTask() misinterpreting exec/AllocEntry return code. amiga.lib
V44.1 fixed this bug, but lots of current programs still have the bugged
CreateTask() routine in them. (nasty bug, causes crashes in low mem
 situations)

Workaround for exec/GetMsg() and 68060 CPUs, braindead programs
calling GetMsg() in tight loop would lock up the system. (minor bug)

exec/ReleaseSemaphore() calculating wrong SS_NESTCOUNT when
both Procure() and ObtainSemaphore() were pending for the same task.
(nasty bug)

graphics.library
================
graphics/InitArea() bug, AreaEllipse() crashed if buffer wasn't
explicitly zeroed & maxvectors was limited to 8191.

The SetRGB32CM() AGA OS call doesn't set the lower 4 bits of the
blue component. You can prove this by calling SetRGB32CM() and
studying the blue component calling GetRGB32().

AmigaOS knows three functions in graphics.library that output
chunky pixels to a RastPort: WritePixelLine8, WritePixelArray8
and WriteChunkyPixels. The original versions of these routines
in the KickstartROM are rather slow and have a bug that trashes
the chunky source buffer. (Michael van Elst)

queue-handler (Heinz Wrobel)
=============
Nasty bugs with PIPE:, like loosing characters.

requestchoice (Joerg Riemer)
=============
The description of Requestchoice explains, that  the  secondary  return  
value  will hold the selected gadget number.  but RequestChoice never
returned a corresponding pr_result2 variable.

the reason for that is requestchoice uses the dos function (setioerr) to save
the  selected  gadget  number as the result2 variable.  but due the fact that
the  result2  variable depends on the returncode a program leaves in register
D0, this one never alifes a succesful exit of requestchoice.

during  the  debugging of RequestChoice i found two other hidden bugs.  there
was  some allocated memory never freed on exit and secondly, the dos function
ReadArgs() was not associated with FreeArgs().

version (Frederic Steinfels)
=======
The original c:version command had a nasty bug: If the last
four bytes of a file contained a $ sign, the version command
went into an endless loop.

amigaguide.library (James Jacobs)
==================
A delay is required between the SetAmigaGuideContext() and
SendAmigaGuideContext() calls.

No AmigaGuideMsg is sent back to the launching application when the
user quits an AmigaGuide which was launched via the
OpenAmigaGuideAsync() function.

asl.library 45.4: (James Jacobs)
===========
If you type a nonexistent path into the "Drawer" gadget (eg. SYS:Foo
when there is no such drawer) and then press ENTER, then (after you have
declined any offer to create the drawer), the program which called the
ASL library will crash (suspend/reboot requester) with error $80000005.

dos.library (James Jacobs)
===========
LoadSeg() will always report success on files that are less than 4 bytes
in size, even though the smallest valid executable is 36 bytes. Files from
4-35 bytes in size are still correctly reported as non-executable.

Installer (James Jacobs)
=========
The documentation for the "askchoice" function is wrong. The result of
"askchoice" is returned as a zero-based ordinal integer, not a bit mask.

version.library (James Jacobs)
===============
It doesn't seem to like being closed if some other program still has it
open (very briefly flashes up some guru alert).

[Gulliver]

@Thomas Richter

Thank you again for taking the time to answer.

I have a couple more of questions:  

1.Will printer.device be 24 bit capable?
2.Is Reaction/ClassAct going to be included?
3.What about DefIcons?
4.AsyncWB?
5.RAWBInfo?
6.Will iconedit support glowicon editing? or will it be the same as 3.1?
7.Will ScreenMode prefs have the "Test" button function like in 3.9?
8.Will WBPattern have a layout option like in 3.9 where it lets you center, scale, tile or scale well the chosen pattern?
9.Mounter tool will be there?
10.A fixed Unarc?

[Gulliver]

I decided to write some preliminary documentation about Prod_Prep for those of you who would like to tinker with it, since it seems from previous posts of this thread, it is widely unknown.

This documentation is full of inacuracies, incomplete, and prone to explode

Prod_Prep v39.1
=========

WARNING: This is an advanced partitioning and format tool. Incorrect usage will definately damage your harddisk structure. You have been warned!

Prod_Prep is the ugly but otherwise powerful brother of HDToolBox. It comes hidden, and is never installed by default. Can be located in the 3.1 floppy disk "Install3.1", inside the C: drawer. It is a very comfortable tool for automatically deploying an entire Amiga system from within a script. It lets you partition and format harddrives in a non interactive manner by feeding the program the proper instructions from a shell/script.

It is still bound to the same limitations that the 3.1 HDToolbox has. All the same warnings about HDToolBox from 3.1 apply to Prod_Prep.

TEMPLATE:


? - list all template commands
Prod_Prep [|-] [device ] [unit ]
   [layout] [badfile ] [formatonly] [noverify] [verifyonly] [slowdown]

quit - exit this program
addpart M|K|C|%|rest
[bootable ] [dostype 0x]
[buffers ] [mask 0x]
[maxtransfer 0x] [customboot ]
[nomount]
deletepart [noerror]- delete a partition
writerdb [force] - write out rigid disk block
format [force] - format drive
verify- check for bad sectors and map out
readfs [] [version ] [dostype 0x] -read FFS from file (default l:fastfilesystem)
synch [on|off] - turn synchronous mode on or off
reselect [on|off] - turn reselect mode on or off
readrdb - reads rigid disk block from drive

Examples:

Prod_Prep layout.script device scsi.device unit 0
; In here, layout.script is a file that should only contain the list of instructions that will alter the drive 0 structure that is connected to scsi.device

; layout.script file contents example:

addpart Workbench: 200M bootable 1 dostype 0x444f5303 buffers 300
addpart Work: rest dostype 0x444f5303 buffers 300
format force
readfs Install3.1:L/FastFileSystem dostype 0x444f5303
reselect on
writerdb force
quit

; End of sample layout.script file (do not add any comments to your script file!).
; This will create 1 bootable, already formatted partition named Workbench: using FFS intl/nodircache of 200MB in size.
; Will also create another non-bootable partition called Work: using FFS intl/nodircache using the rest of the drive remaining space.
; The filesystem used will come from Install3.1:L/FastFileSystem

It is important for the correct use of this tool to understand that each single filesystem buffer will take away 1KB of ram from your system, so if you analyze the example above, you will realize, it will cost you 600KB of ram. The more buffers the faster your system will be able to handle big files, always at the cost of precious ram. As a suggestion, never go too low, to avoid potential issues, and if your system can handle the loss of 2MB of ram, just go nuts, and go for up to 2000 buffers. An average between 90 and 300 is what I would consider a good compromise between speed and ram usage. But the choice is yours, and it may largely depend on how your system is setup/used.

ADDENDUM: AMIGA FILESYSTEM DOSTYPES

FILESYSTEM, DOSTYPE, COMMENTS, LIMITS
PFS3_aio, 0x50465303, Amiga PFS file system 3. Even runs on a 68000 with kickstart v1.3!, Filename 107 chars/filesize 2GB/partition 104GB.        
PFS3_aio, 0x50445303, Amiga PFS file system 3 SCSIdirect. Even runs on a 68000 with kickstart v1.3!, Filename 107 chars/filesize 2GB/partition 104GB.
SFS0, 0x53465300, Amiga Smart File System V1, Filename 107 chars/filesize 2GB/partition 128GB.
SFS2, 0x53465302 Amiga Smart File System V2, Filename 107 chars/filesize 2GB/partition 1TB.
FFS Intl+noDirCache, 0x444f5303, This is the most commonly used and compatible one. Came with AmigaOS >= 2.0., Filename 30 chars/filesize 2GB/partition 2GB.  
FFS2, 0x444f5307, This is the one that comes with OS4. Also supports DOS 8., Filename 107 chars/filesize 2GB/partition 128TB.
FFSAIO, 0x444f5308, DOS 8. Beta status. It comes as a patch for FFS v45.13 (FFS 45.20r1)., Filename 54 chars/filesize 2GB/partition 2GB.

The scsi.device (or whatever name the harddrive interface driver has on your expanded system), has limitations of its own, and those will be imposed first, before, the ones the filesystem of your choice has.

There are many other Amiga filesystems. This does not pretend to be a complete list. Most of the missing ones, are not worth mentioning since they are not common, too old, or buggy for the average user.

[Gulliver]

It seems that the entire HDD management code is hanging on some nasty code leftovers and bizarre workarounds, waiting to fall apart any day.  

Not a comfortable way to code, I grant you that, and applaud you both for your bravery.

[Gulliver]

Validation: The mother of all evils

One of the major setbacks AmigaOS 3.1.x users will face, is that along with the benefits of large drives and newer FFS support, big volumes will definately become very common. This means that if you had to wait for quite some time before to get your volumes validated, on lets say a 500MB partition, just imagine how long will it take with a 20GB partition (and I am being really modest here).

So if in 2018 you buy, lets say for the purpose of this exercise, a new 2TB harddrive and install FFS (DOS7) on it, if a FS error occurs you will have to wait till your grandchildren finish college with your Amiga 1200 turned on all that time, to get the volume validated.

I know, clever users will still partition drives in a more reasonable manner, so that volumes dont get that large, but then is that a real solution?

And please dont tell me this is no problem, as this has been happening for decades, despite all AmigaOS developers not visualizing it as an issue. A simple fact, that illustrates the severity of the matter: many, if not all of the best third party filesystems for Amiga were originally developed to avoid the validation paradigm (PFS, SFS, etc), and they were quite succesfull, so much that many are still used and updated till this day, and are despite of some unresolved issues, still chosen over FFS which comes already installed by default.

Whilst keeping the status quo, and ignoring the issue, is the cheapest way, it will bring up lots of issues, especially with inexperienced users. Some flexibility could be certainly adopted regarding FFS. My proposed solution is a cli command to tweak FFS behaviour:

C:Validate

Changes how the filesystem manages write errors.

OPTIONS

oneatatime -Validates/fixes volumes one at a time.
novalidate -Ignores any FS errors and doesnt validate/fix them.
prevent    -Attempts to trap the reset signal so that FS write corruption may be prevented.
warn       -Detects FS errors and gives a warning but doesnt perform any repair/validation action at all.
fix        -This is the default FS behaviour: If FS error occurs, all volumes are mounted read only and validate/repair starts on all of them.
waitfix    -If FS error occurs all volume activity is halted until validate/repair is finished. Then volume activity resumes as usual.
ask        -If error in the FS occurs, the user is asked to either validate/repair, ignore or mount readonly.

some options can be combined:

the "prevent" option can be combined with any other one.
"oneatatime" can be combined with "waitfix", "ask" and obviously as mentioned before, with "prevent".

[Gulliver]

@Thomas Richter
@olsen

A journaling filesystem woulld certainly improve reliability.

But why not better kill more birds with one stone?

A versioning filesystem would be a much better overall solution, because not only it would allow the user to roll back to previous versions of files in case of system corruption, but it also means that if the filesystem does the work twice (comitting to disk), you at least do the job completely and not halfway like in a journaled filesystem, and also keep the benefit of having an older backup automatically.

The drawback, is of course, increased disk space usage, but is that really a problem in an Amiga system where files are really small in size and when storage media keeps getting bigger each day? And of course, you can also provide some algorithm for garbage collection when the amount of backup exceeds certain amount to free some space.

As a huge side benefit, a versioning filesystem will also be an excellent filesystem candidate for flash/ssd solutions. We dont have any filesystem or tools to deal with wear levelling on Amigaland. A versioning filesystem does not need to overwrite the same blocks of data (unlike a journaled one). And you can make each block/cluster keep a write counter to estimate the media life and take action when reaching a critical life cycle limit.

Another benefit, is that due to the nature of the versioning filesystem, it may be used, with little customization as a developer aid.

[Gulliver]

@olsen

It is true, a journaled filesystem is much more straightforward in its design and much easier to implement, besides, it doesnt require a change in the API. Whilst a log or versioning filesystem would be great, I understand the amount of challenge it would represent in an Amiga enviroment is well into the realm of uncharted territory.

Sometimes it is better to just do what you know best.

[Gulliver]

Yes, it would nice to know how is it going.

[Gulliver]

I would say this is cancelled.

The coder now has a full-time job trash-talking Apollo/Vampire stuff, so there's no time left to actually produce something himself I'm afraid

Well, then I am glad, because now the Apollo guys will be forced to implement a proper MMU tu run Linux. Or they could stil resell AmigaOS 3.9 and Fusion mac emulation.

[Gulliver]

C:Edit was probably the last executable program in BCPL. The last handler in BCPL was the Aux-Handler, but that got rewritten already. My current handling of C:Edit is that I removed it from the distribution. It is about the only binary (along with C:MagTape) I haven't used *once* in 20 years. It shares now the same destiny with MagTape -> NIL:

It is a pitty as c:edit is great for building scripts to edit the startup-sequence in a non interactive manner. I am all for a replacement, but not for a removal without something that replaces that functionality.

We had a little chat about edit a while ago about this with Olaf Barthel:
http://www.amiga.org/forums/showthread.php?t=57601&

On the other hand, I have nothing good to say in favour of Magtape.

[Gulliver]

C:Wait and C:Sort, both version 42.1, at least the sources are not a problem, hehe

The FILE option of Wait 42.1 has turned out to be quite useful for me, simple trick to solve situations with several background running scripts having dependencies on each other etc.

The problem with those updates as far as I know is that they are post 3.1 CBM era and are not part of the rights Hyperion has, so they cant be included, but of course, their extra features can be reimplemented.

[Gulliver]

Code: [Select]


dr-xr-xr-x  0 root   root        0 Jul 19  1993 os-source/v42/src/workbench/c/wait/
-r-xr-xr-x  0 root   root     5628 Jul 19  1993 os-source/v42/src/workbench/c/wait/wait.ld
-r-xr-xr-x  0 root   root        2 Jul 12  1993 os-source/v42/src/workbench/c/wait/wait_rev.rev
-r-xr-xr-x  0 root   root      217 Jul 12  1993 os-source/v42/src/workbench/c/wait/wait_rev.i
-r-xr-xr-x  0 root   root      976 Jul 19  1993 os-source/v42/src/workbench/c/wait/wait.ld.strip
-r-xr-xr-x  0 root   root     5604 Jul 19  1993 os-source/v42/src/workbench/c/wait/wait.o
-r-xr-xr-x  0 root   root     1158 Jul 19  1993 os-source/v42/src/workbench/c/wait/wait.map
-r-xr-xr-x  0 root   root      175 Jul 12  1993 os-source/v42/src/workbench/c/wait/wait_rev.h
-r-xr-xr-x  0 root   root     6050 Jul 12  1993 os-source/v42/src/workbench/c/wait/wait.c
-r-xr-xr-x  0 root   root     2201 Jul 19  1993 os-source/v42/src/workbench/c/wait/lmkfile
dr-xr-xr-x  0 root   root        0 Aug  9  1993 os-source/v42/src/workbench/c/sort/
-r-xr-xr-x  0 root   root     1884 Aug  9  1993 os-source/v42/src/workbench/c/sort/sort.map
-r-xr-xr-x  0 root   root     5984 Aug  9  1993 os-source/v42/src/workbench/c/sort/sort.ld
-r-xr-xr-x  0 root   root      214 Aug  9  1993 os-source/v42/src/workbench/c/sort/sort_rev.i
-r-xr-xr-x  0 root   root    18533 Aug  9  1993 os-source/v42/src/workbench/c/sort/sort.c
-r-xr-xr-x  0 root   root     2280 Aug  9  1993 os-source/v42/src/workbench/c/sort/sort.ld.strip
-r-xr-xr-x  0 root   root     2201 Aug  9  1993 os-source/v42/src/workbench/c/sort/lmkfile
-r-xr-xr-x  0 root   root      172 Aug  9  1993 os-source/v42/src/workbench/c/sort/sort_rev.h
-r-xr-xr-x  0 root   root        2 Aug  9  1993 os-source/v42/src/workbench/c/sort/sort_rev.rev


From what I recall - CBM folded in 1994.

https://pastebin.com/L3uU1Cmq

The key element there is not that of the year, but of the version number.
Version 42.x was post 3.1, so it is a no go, despite still being built during CBM era.