Hyperion announces OS 3.1 update

[guest11527]

When it comes to the FFS, one man's adventure is another man's daredevil stunt with tragically comic consequences.

Yup. Unless Toni invested quite a bit of work, this will not do. About the only thing that was present of DOS/8 in the 3.9 release is ExAll() and friends, and the computation of the maximum file size. Everything else on file names in DOS/8 was there borken... Open, Rename, Lock, .... you name it. I cleaned this up for V46, so it now works quite fine on my HD since probably a year or so.

One way or another: DOS/8 is most likely not going to become official. Not because it is too experimental - rather because we have another alternative that is more flexible.

[olsen]

Oh well, it really depends on what you are doing. As far as raw data transfer is concerned, neither file system can do any magic. It just has to take the user blocks, and direct the device to write them out. So far, so simple.

The things which the FFS does well it really does better than any other Amiga file system. For example, the FFS performs asynchronous read operations with very little overhead, and nearly zero CPU load if the underlying storage system supports DMA. There's even a demo created by Commodore which shows off streaming a pre-rendered animation (generated by "VistaPro"), the use of the double-buffering screen API in graphics.library, and rendering animations on top of that.

However, what the FFS does well is a small subset of the operations a file system generally has to get done properly, because they occur so frequently. This is where the FFS does not compare well against other Amiga file systems, except perhaps CrossDOS or CDFS (irony be damned).

What ought to work well is storage management, as in finding free space to store new data in quickly, while at the same time keeping fragmentation at bay and just maybe reduce seek latency at the same time (releasing storage space to the free pool is the flip side of this coin). This is doable, but it will cost you, e.g. XFS does this very well, but it has to throw a lot of memory and storage space at the effort.

Directory scanning, including retrieval of metadata, ought to be fast, too. The FFS has never scored well here, because it does not separate directory lists and file metadata. You should not have to wait for the directory contents to arrive in dribs and drabs. It makes the floppy/hard disk appear to be much slower than it actually is. The original 1.x Workbench tried to sidestep this problem by recording the contents of the directory in a ".info" file, but it only went so far: instead of having to walk through the entire directory list and reading the icon files it found, the ".info" file only helped to cut the scanning process short. Finding and reading the icon files remained dead slow because the file system didn't care about seek times or fragmentation.

I wish there were some convenient remedies in reach to address the shortcomings of the FFS design, but the only solutions I can imagine right now begin with introducing journaling with write-ahead logging. Once that can be made to work, we could enable dircache mode, which would instantly improve the directory access performance. Making storage management work better is particularly difficult because there exist no data structures in the FFS design to assist the process.

In any case, resolving these limitations will have the probably unwelcome side-effect of requiring more RAM and more storage space than the FFS needs today. Could be this will be a worthwhile trade-off, but none of these possible changes will come together without a lot of prior work and testing. We are still kinda short-handed in brushing up AmigaOS 3.1 for more exciting things to come, so all of the FFS changes might have to wait a little while longer.

[Gulliver]

Hi, I have a couple of questions regarding the 3.1 update:

Will there be a CD32 workbench version support? Or will CD32 users still have to resort to hacks and custom built workbench cdroms? (freeanim+joymouse+setcdreboot+cdaudio+OSK)

And then, what about also adding CTDV workbench support with the RMTM command?

HDToolbox unfriendly brother prod_prep will be updated to support large drives?

Will you also include the quite handy but never directly included "reboot" and "findresident" commands?

Thank you for your time.

[chris]

Once that can be made to work, we could enable dircache mode, which would instantly improve the directory access performance. Making storage management work better is particularly difficult because there exist no data structures in the FFS design to assist the process.

Is that a different dircache mode to Commodore's one?  IIRC that had some shortcomings and wasn't recommended to be used, although I forget why.

[olsen]

Hi, I have a couple of questions regarding the 3.1 update:

Will there be a CD32 workbench version support? Or will CD32 users still have to resort to hacks and custom built workbench cdroms? (freeanim+joymouse+setcdreboot+cdaudio+OSK)

Sorry, this has not been discussed yet. Building a CD32 ROM should be possible with a minimum of fuss, using the CD32-specific components last updated in 1994. But so far nobody has even investigated if it would be possible to build them from source.

So, you might want to wait until we've shipped something and just maybe we could have a look at the CD32 bits to see what can be made to build. This is rather exotic code to begin with, mind you...

And then, what about also adding CTDV workbench support with the RMTM command?

The CDTV code is closely linked to the Kickstart 1.3 ROM code. Nobody knows how to build any of this today, except perhaps the original CDTV project members, including Carl Sassenrath, and we haven't asked them yet. So, the short answer for now is "probably not, because we haven't got enough of a clue to make it happen".

I for one would like to see more light shed on the CDTV code, which is frankly pretty amazing stuff which was so far ahead of its time that it was almost tragically destined not to achieve lasting impact on the state of the art.

HDToolbox unfriendly brother prod_prep will be updated to support large drives?

Sir, I bow to you, because you just made my day merely by asking that very question

Yes, "prod_prep" has been updated by yours truly to support large drives, to the general puzzlement of everybody involved in this AmigaOS 3.1 update project (most Amiga users don't know what "prod_prep" does, some are probably afraid to ask, or embarrassed to admit having wrecked their hard disks by accidentally using it). However, no amount of rewriting can remove all the quirky & twisted limitations of that runt of a partitioning program. Right now "prod_prep" is generally safe not to make a big mess/crash/set your cat on fire if it encounters a drive larger than 2 Gigabytes. If my calculations are correct, the upper limit are 2.2 Terabytes, but you would be very "brave" indeed to allow "prod_prep" to set up a partition scheme for you on a disk that large.

We really ought to have a safe and sound baseline RDB library which would support all RDB data structure management operations, including reading and writing them. A robust "prod_prep" and "HDToolBox" could be built from that library, and hard disk controllers which deal with RDB data could rely upon a single tested library for that purpose. Right now we have none of that, with hilarious consequences.

Will you also include the quite handy but never directly included "reboot" and "findresident" commands?

These are part of the "Install3.1" disk "C" commands, if I remember correctly, so in a way these are still included, just not installed by the installation script. I believe you would like to see them installed among the regular shell commands, is that correct?

[olsen]

Is that a different dircache mode to Commodore's one?  IIRC that had some shortcomings and wasn't recommended to be used, although I forget why.

It's the same thing, and just to confuddle and befuse everybody, it has several different names (DCFS, fastdir mode, dircache mode).

The shortcomings are finally documented in the Wikipedia article (https://en.wikipedia.org/wiki/Amiga_Fast_File_System), which links to the low level documentation on the DCFS and LNFS flavours which I wrote so that finally there was something that Wikipedia could quote

[guest11527]

It's the same thing, and just to confuddle and befuse everybody, it has several different names (DCFS, fastdir mode, dircache mode).

Oh no, not DCFS again... Yes, this thing "sort of" works, but I was more than happy that I did not need to touch *this* part of the code for the new DOS variants. Mind you, the FFS code is not in perfect shape. It's the sort of assembler code that lacks proper convention - in the sense that *this* function requires arguments in a3 and d0, and messes a2, but *that* function preserves registers and requires arguments in a0 and d0. Not that anyone felt that documenting calling conventions would be necessary...

One way or another: I will certainly, most definitely, not touch the DCFS code on *this* particular implementation.

[guest11527]

Will there be a CD32 workbench version support? Or will CD32 users still have to resort to hacks and custom built workbench cdroms? (freeanim+joymouse+setcdreboot+cdaudio+OSK)

The build chain includes CD32 targets, yes. Question is whether there will be any chance to test it.

And then, what about also adding CTDV workbench support with the RMTM command?

...and there are no traces of CDTV support in the build chain whatsoever. So probably make this a "likely not".

HDToolbox unfriendly brother prod_prep will be updated to support large drives?

Hah, and I didn't even knew about what that is probably a day ago. (-: Yes, Olaf updated prod_prep. It will probably go through another polishing stage to make it use TD64 or DirectSCSI if applicable, but it's again the type of source you better don't touch, and you feel dirty all over after having done the job.

Will you also include the quite handy but never directly included "reboot" and "findresident" commands?

I cannot really tell, as I haven't checked whether we have sources. But it looks to me as they are either easy to do, or easily replacable by existing tools.

[kolla]

While we are throwing around ideas... how about some mechanism to have system time survive reboots, even without RTC available?

Would also be nice if C:reboot could "lock" mounted local disk devices before performing the reboot (I think for example LoadModule does this).

C:Date needs some love, it's confusing the way it does locale, and it could really need "LFORMAT" like option.

And speaking of LFORMAT, the default output from C:List does not work well with long filenames and more, would make sense to make it terminal/console width aware, or ar least user configurable (for example via a variable). Oh, and.. the lack of a %X (where X is any available letter) that is similar to %P only without trailing / has been driving me nuts!

[magnetic]

Great, but I just installed OS3.9 with all possible updates . How much more someone can update 060.library?

I think people would be happy to have new chip rom with "official" updates so we don't need to restart Amiga's multiple times.

We need new OS3.95 .

What about OS3.9 update...

They dont have rights for "3.9" haage and parnter still retains them afaik

[ferrellsl]

They dont have rights for "3.9" haage and parnter still retains them afaik

Rights extended to them by whom?  A long dead company?  Any agreements they had with a company or companies that are now out of business are null and void and I'm certain they were never extended a perpetual license, so stop spreading this crap.

OS3.x became abandoware years ago yet Cloanto kept distributing copies anyway....as have other dealers out there who thought they could make a quick buck by continuing to sell copies....and now Hyperion has jumped on the same bandwagon.  They have no more rights to OS3.x than anyone else...they just have an attorney who threatens to sue anyone who wants to sell copies.

[olsen]

Rights extended to them by whom?  A long dead company?  Any agreements they had with a company or companies that are now out of business are null and void and I'm certain they were never extended a perpetual license, so stop spreading this crap.

Haage & Partner licensed or aquired code from third parties which is part of the AmigaOS 3.5/3.9 update. They also created code in house which is part of AmigaOS 3.5/3.9, and they also paid developers to create code for it. The company still owns this software and, to the best of my knowledge, has not licensed or sold it yet.

None of this specific code owned by Haage & Partner is tangled up with the Amiga International-licensed AmigaOS 3.1 code. Even if Haage & Partner wanted to, they could not "bake" a fully legally sanctioned AmigaOS 3.9 out of their properties and AmigaOS 3.1.

Yes, it sounds weird, because it is. This is the Amiga business, after all

[guest11527]

While we are throwing around ideas... how about some mechanism to have system time survive reboots, even without RTC available?

Yes, I noted the issue, but haven't thought about a cure yet. There are so many other items on my agenda, it is hard to keep an overview.

Would also be nice if C:reboot could "lock" mounted local disk devices before performing the reboot (I think for example LoadModule does this).

This is all fairly trivial stuff. DiskSafe + Reboot will do just that for you, which puts another item on my agenda. It would be much better if DiskSafe would be integrated into the  FFS instead of operating as a patch on top of the dos.library. IOWs, I don't quite see why existing tools cannot do that for you right away.

C:Date needs some love, it's confusing the way it does locale, and it could really need "LFORMAT" like option.

Maybe yes. This issue wasn't pressing enough to be on my list.

And speaking of LFORMAT, the default output from C:List does not work well with long filenames and more, would make sense to make it terminal/console width aware, or ar least user configurable (for example via a variable).

List is a rather simple "linear list" output I don't want to modify for backwards compatibility reasons. There are surely programs that parse what list generates, so I'd better not touch it. Yes, it has problems with long file names, but more internally in the buffer management. They have been fixed. List also got an option to indicate links and link targets. Actually, the same as in AmigaOs 4.0 (don't create too many inconsistencies here).

If you want a "nicely formatted output", this is what "dir" is good for. Yes, "Dir" got some care, and yes, it is now aware of the console dimensions. It will even work nicely over AUX: provided the connected terminal speaks CSI sequences. It no longer uses only two columns, but computes the number of columns dynamically. Pretty much like "ls" on *ix does.

Dir also learned softlinks and hardlinks.

Oh, and.. the lack of a %X (where X is any available letter) that is similar to %P only without trailing / has been driving me nuts!

This sounds like a doable idea. I just need to check which letter is available, and whether there is something like this in Os 4.

[kolla]

Thanks for answers

[Gulliver]

@olsen
@Thomas Richter

Thank you both for explaining the difficult situation regarding both the CDTV and the CD32.

I believe prod_prep is often forgotten, underlooked and unapreciated, and it is a very powerful tool that all amiga users should know about. Dealing with it, is certainly not easy, and could be made a little bit less difficult by including an amigaguide manual with plenty usage examples.

I personally use prod_prep for deploying my own customized system/workbench to my amigas from within a simple script. It is very small and it is a formidable tool for emergency install floppies, where space is severely restricted. This way you save space for other important components and dont need to put both format and hdtoolbox which are quite large compared to prod_prep. Furthermore, it is a bonus that you are not required to use it in an interactive way and that you can fully automate it.

An RDB library sounds like a very good idea.

Will you please replace graphicdump with a more usable snapshot tool?
It is archaic and not flexible at all. A simple screen grabber that saves to disk would suit much better, and even more now, taking into account the sad state of the printing system.

What about picture.datatype? Do you still have the v45 sources? And even a v45 rebuild without the infamous StormC compiler would be apreciated. I dont enjoy the fat binary aspect of its last incarnation in 3.9. I dont want to deal with code that I wont use. But I still want the 24 bit support.
And of course, I wont even need to mention all the datatypes that need updates. It seems a lot of work is here waiting to drive you crazy.

Regarding audio.device: do you have some plans to somehow incorporate AHI into it, so that old applications can finally become more friendly with soundcards? Afterall, unlike the RTG scenario, AHI is open source and it has become the unique and default standard for modern audio applications with no other counterpart attempting to replicate that on the Amiga.

Speaking about usefull commands, yes, my idea was to have both reboot and findresident installed. BTW, will you also include "warnifpressed". I believe you have the sources for all of these.

Thanks again.

[ferrellsl]

Haage & Partner licensed or aquired code from third parties which is part of the AmigaOS 3.5/3.9 update. They also created code in house which is part of AmigaOS 3.5/3.9, and they also paid developers to create code for it. The company still owns this software and, to the best of my knowledge, has not licensed or sold it yet.

None of this specific code owned by Haage & Partner is tangled up with the Amiga International-licensed AmigaOS 3.1 code. Even if Haage & Partner wanted to, they could not "bake" a fully legally sanctioned AmigaOS 3.9 out of their properties and AmigaOS 3.1.

Yes, it sounds weird, because it is. This is the Amiga business, after all

Ah, that makes perfect sense.  So until there's a clear owner of OS3.1 and the IP associated with it, Haage & Partner can't release and distribute a full-blown update of OS3.9.  I'm not sure that this issue will ever get lined out.  There are just too many vultures fighting over the dead carcass of OS3.1 and they fail to realize that there isn't a significant amount of money to be made even if they gained said rights.  Even with the Vampire's popularity I would guess that there would be fewer than 100 buyers of a Vampire specific release of OS3.1.  Most Vampire owners already own a copy of OS3.1 or 3.9 and they're quite happy to just patch their existing copies as needed.

Dreaming of an Amiga revival where Vampires are being sold in the tens of thousands bundled with copies of OS3.1 licensed from Hyperion borders on the absurd, but delusional business plans run rampant in this hobby that encompasses dead hardware and dead operating systems....