Welcome, Guest. Please login or register.

Author Topic: Os 3.1.4 - List of bug fixes and changes by component  (Read 84933 times)

Description:

0 Members and 2 Guests are viewing this topic.

Offline Gulliver

Re: Os 3.1.4 - List of bug fixes and changes by component
« Reply #89 on: November 13, 2018, 05:54:46 PM »
@Thomas so is it even necessary to have the maxtransfer field available? If it's not necessary for the user to change it shouldn't the field to edit it just be removed?

We've been trained over the years to diddle with maxtransfer for various devices. If it is handled automatically now, if a user doesn't read the documentation before using (I know, I know, but that's another discussion), wouldn't having it available for them to diddle with just cause problems?

I guess the questions that need answering to make it clear follow. Are there ANY circumstances with 3.1.4 that maxtransfer should be edited? Is it the case that whatever is entered is ignored since it is handled automatically now by Kickstart / devices?

MaxTransfer is not an issue on scsi.device. Many third party scsi devices do need it.  ;D
 

Offline kolla

Re: Os 3.1.4 - List of bug fixes and changes by component
« Reply #90 on: November 13, 2018, 09:14:32 PM »
And if one has multiple incarnations of scsi.device in use on same system, such as booting with kick 3.1 before loading scsi.device v45, or when using IDEFix, one better stay safe than sorry. Especially with "exotic" implementations of Gayle.
B5D6A1D019D5D45BCC56F4782AC220D8B3E2A6CC
---
A3000/060CSPPC+CVPPC/128MB + 256MB BigRAM/Deneb USB
A4000/CS060/Mediator4000Di/Voodoo5/128MB
A1200/Blz1260/IndyAGA/192MB
A1200/Blz1260/64MB
A1200/Blz1230III/32MB
A1200/ACA1221
A600/V600v2/Subway USB
A600/Apollo630/32MB
A600/A6095
CD32/SX32/32MB/Plipbox
CD32/TF328
A500/V500v2
A500/MTec520
CDTV
MiSTer, MiST, FleaFPGAs and original Minimig
Peg1, SAM440 and Mac minis with MorphOS
 

guest11527

  • Guest
Re: Os 3.1.4 - List of bug fixes and changes by component
« Reply #91 on: November 14, 2018, 06:48:39 AM »
@Thomas so is it even necessary to have the maxtransfer field available? If it's not necessary for the user to change it shouldn't the field to edit it just be removed?
No, because the HDToolBox does not only handle the scsi.device, but also other devices, and I can of course make no statement how they handle the situation. Note well, it is not "handled automatically" by anything. All I'm saying is "the scsi.device does it right now". I'm not saying "all other devices do it right now". As far as "correct devices" are concerned, the omniscsi is certainly also correct.

Once again, "MaxTransfer", "Mask" and "BufMemType" are workarounds for broken devices, third party or not. The scsi.device got "unbroken", but I cannot make any statements beyond that.

I guess the questions that need answering to make it clear follow. Are there ANY circumstances with 3.1.4 that maxtransfer should be edited? Is it the case that whatever is entered is ignored since it is handled automatically now by Kickstart / devices?
Check with the documentation that came with the host adapter you are using. The CBM devices do not need it, the GuruROM does not need it. Everything else... well, anybodies guess.
 
The following users thanked this post: Tygre, Pgovotsos

guest11527

  • Guest
Re: Os 3.1.4 - List of bug fixes and changes by component
« Reply #92 on: November 14, 2018, 06:51:47 AM »
And if one has multiple incarnations of scsi.device in use on same system, such as booting with kick 3.1 before loading scsi.device v45, or when using IDEFix, one better stay safe than sorry. Especially with "exotic" implementations of Gayle.
Once again, just to make this completely sure: This has nothing, absolutely nothing to do with Gayle. Gayle is just a bunch of latches, not more. Either you get a PATA command through, or you don't, but there is no situation in which one version of Gayle requires another MaxTransfer than another - this is simply not possible.

It is just a question of the PATA aka IDE protocol, and its proper implementation. The V45 scsi.device is more careful in its implementation, and handles this race condition of >255 sectors gracefully. Regardless which version of Gayle is installed. Gayle is completely innocent in this respect.
 
The following users thanked this post: Tygre

Offline NinjaCyborg

Re: Os 3.1.4 - List of bug fixes and changes by component
« Reply #93 on: November 19, 2018, 05:58:10 PM »
Dear Thomas please continue your notes, and ignore the trolls. Much much thanks to you for your hard work and contributions over many years.
 

guest11527

  • Guest
Re: Os 3.1.4 - List of bug fixes and changes by component
« Reply #94 on: November 20, 2018, 03:52:20 PM »
Sorry, I know I'm late on this. But I recently moved, and my former ISP does not deliver to my new appartment, and the new one doesn't get things going. It's supposed to take 4-6 weeks until I'm back online (German efficiency...), meaning probably not before Christmas. And I cannot waste the time of my employer overly.
 
The following users thanked this post: Tygre, LoadWB

Offline paul1981

Re: Os 3.1.4 - List of bug fixes and changes by component
« Reply #95 on: November 22, 2018, 05:43:18 PM »
Dear Thomas please continue your notes, and ignore the trolls. Much much thanks to you for your hard work and contributions over many years.

I've read through this whole thread, and in all honesty I don't believe there is any trolling going off here. Kolla has asked some valid questions in my mind and I was happy to see Thomas answer them.

The only MaxTransfer issue I know of was a problem with CF cards attached to the internal on-board IDE controllers of the A600, A1200 and presumably the A4000. I only occasionally use CF cards for file transfer via PCMCIA so it's a non-issue for me anyway. Regardless, the fix was easy as it just required you manually specifying a MaxTransfer value of 0xFE00 in HDToolbox.
 

guest11527

  • Guest
Re: Os 3.1.4 - List of bug fixes and changes by component
« Reply #96 on: December 08, 2018, 09:43:12 AM »
So, internet is back, it "only" took a month for my new ISP to setup the line. But it is "fibre to home", so I cannot complain. Hence, let's continue here, even though I might have forgotten where I stopped.

Shell Pipes: We had this already. It is an old feature in a new interface - there is no longer the Pipe command because the shell handles it all by itself. The advantage is that there is no need to go through a second-step parsing, which is particularly tricky with the awkward shell language. Depending on the Pipe command you used, some recognized tokens like STDIN: or STDOUT: to access directly the input or output stream. Note well, there is no "device" named STDIN: or STDOUT:, this was simply "guessing and token recognition", to allow such things as
Code: [Select]
list | type stdin: hex
which would give a directory listing in hex. Don't ask what this is good for, it is just the simplest example I could think of. "Type" does not support "reading" from stdin, so you had to use this token.

With 3.1.4, there is no "Pipe" command, and there is no longer any magic token that represents a stream. But you can do the same by similar means, by a feature of the qeue-handler. If it is opened without any file name, it rather reads or writes (depending on how it is opened) from the currently active input or output pipe.

Sounds complicated? Not really, it is quite straightforeward. The above converts to:
Code: [Select]
list | type pipe: hex
Hence, list pipes its output to the pipe, and "type" picks it up there, via "PIPE:". There is no ambiguitity, "PIPE:" is context-sensitive, i.e. you can run the same in two shells simultaneously without any chance of confusion because "PIPE:" knowns where the input or output comes from or goes to, and PIPE: also knows whether it should operate from the reading or writing end of the pipe because the program opens the stream either for reading or writing. In the above case, for reading of course, so it connects to the output of list.

Morale of the story: Just replace STDIN: or STDOUT: (if your "pipe command of the day" supported that) by PIPE: and all will be good.
 
The following users thanked this post: Tygre

Offline kolla

Re: Os 3.1.4 - List of bug fixes and changes by component
« Reply #97 on: December 08, 2018, 05:39:57 PM »
How about $_phar and $_mchar? Still used, or are things now hard coded to | and ||?
(I have typically used && for the latter)

Is CONSOLE: used by any chance? I've been asked to insert CONSOLE: a few times with shell v46.
B5D6A1D019D5D45BCC56F4782AC220D8B3E2A6CC
---
A3000/060CSPPC+CVPPC/128MB + 256MB BigRAM/Deneb USB
A4000/CS060/Mediator4000Di/Voodoo5/128MB
A1200/Blz1260/IndyAGA/192MB
A1200/Blz1260/64MB
A1200/Blz1230III/32MB
A1200/ACA1221
A600/V600v2/Subway USB
A600/Apollo630/32MB
A600/A6095
CD32/SX32/32MB/Plipbox
CD32/TF328
A500/V500v2
A500/MTec520
CDTV
MiSTer, MiST, FleaFPGAs and original Minimig
Peg1, SAM440 and Mac minis with MorphOS
 

Offline TribbleSmasher

Re: Os 3.1.4 - List of bug fixes and changes by component
« Reply #98 on: December 08, 2018, 05:50:34 PM »
Is CONSOLE: used by any chance? I've been asked to insert CONSOLE: a few times with shell v46.
If this could turn up in any file requester: please don't. I hate "fake" drives in them, seriously.
 

guest11527

  • Guest
Re: Os 3.1.4 - List of bug fixes and changes by component
« Reply #99 on: December 08, 2018, 05:53:14 PM »
How about $_phar and $_mchar? Still used, or are things now hard coded to | and ||?
(I have typically used && for the latter)
No, they are hardcoded. The shell syntax is complicated enough as it is. Note that it is the recommended setting anyhow and agrees with the ARexx syntax.

Is CONSOLE: used by any chance? I've been asked to insert CONSOLE: a few times with shell v46.
CONSOLE: is used for &. In particular, the runback token uses named consoles to allow proper job control. Note a delicate difference between CONSOLE: and *. The latter is the "current console owner = the current job", "CONSOLE:" is the default console owner, i.e. the job that launched the shell. "&" uses "CONSOLE:<a unique token>" so that job control would work for consoles that distinguish between jobs.
 
The following users thanked this post: Tygre

Offline kolla

Re: Os 3.1.4 - List of bug fixes and changes by component
« Reply #100 on: December 09, 2018, 10:38:22 AM »
I see. Yes, guess I may have attempted using & in some temporary script, which then has been launched with "run >NIL: script" in the background, detaching it from any console capable of job control. Using & is more equivalent to "run >console:" I suppose.
B5D6A1D019D5D45BCC56F4782AC220D8B3E2A6CC
---
A3000/060CSPPC+CVPPC/128MB + 256MB BigRAM/Deneb USB
A4000/CS060/Mediator4000Di/Voodoo5/128MB
A1200/Blz1260/IndyAGA/192MB
A1200/Blz1260/64MB
A1200/Blz1230III/32MB
A1200/ACA1221
A600/V600v2/Subway USB
A600/Apollo630/32MB
A600/A6095
CD32/SX32/32MB/Plipbox
CD32/TF328
A500/V500v2
A500/MTec520
CDTV
MiSTer, MiST, FleaFPGAs and original Minimig
Peg1, SAM440 and Mac minis with MorphOS
 

guest11527

  • Guest
Re: Os 3.1.4 - List of bug fixes and changes by component
« Reply #101 on: December 09, 2018, 11:11:52 AM »
I believe I've already provided some details on CrossDos, but nevermind doing it again and going a bit more into depth. CrossDos is also pretty much new, at least many major parts have been exchanged, and there is unfortunately still one bug left we need to fix in a follow-up release (tentatively, 3.1.4.1). The big picture is that CrossDos is now multithreaded, same as the FFS, and it can handle large volumes (though, due to the bug, not long FATs). That means, it supports FAT16,FAT24 and (that is the new part) FAT32. It also includes support for long file names, something called "VFAT" in the Linux world. Last M$ patents on this stuff run out Febrary last year, so we are safe.

Multithreaded means that CrossDos can fire off an I/O operation such as reading a long file, it can still respond to other incoming requests. As long as the requests can be handled from cache - and CrossDos does have a cache - they can be answered immediately. Previous CrossDos releases tried this in a clumpsy way, by always starting two processes. One handled "ACTION_DISK_INFO", and forwarded all other requests to a second task. This would at least avoid slowing down the frequent disk check by the workbench which triggers exactly this packet, but doesn't help much otherwise. New CrossDos is a single process, not two, but it uses a technique called "coroutines", something it shares with FFS and the CD File system (more on this in another episode).

For normal routines, you have subroutines A, B, C where A calls B, then B calls C, C returns to B, and B returns to A. Coroutines work different. From A's perspective, A calls B, and - from inside B - B calls A. A typical example for this (and this is how it works in just another project of mine, ViNCEd) is parsing of CSI sequences, e.g. CSIx;ym to set the font color and font attributes in the console. Here we have one function B that calls A to get a new character, then, if this is a CSI or ESC, it reads the next - ok, is it a number? - parses the number, then continues parsing to the next semicolon, then when the final letter arrives, it has all the parameters it needs to collect, and calls the corresponding lower-level function to set the attributes. So, the parser - here B - needs the data letter by letter or digit by digit. Unfortunately, this is not the way how the data arrives. Instead, the user writes data into the console, and probably not even in complete sequences. There is nothing wrong writing the CSI with a single write call, then writing the digits in a second call, and then writing the final letter in a last call. So, from that perspective, the function that receives all the request from the user - and this is A - gets data in chunks, and then calls into B. So from the perspective of the dos.library, the top-level is A, which collects user requests, and calls into the parser B. From the parser perspective, B namely, it needs to call A to retrieve the next character. Clearly, this doesn't work with classical subroutines, because either A or B would need to be turned "inside out".

There are two solutions for this problem. One solution is "state machines", which are hard to debug and complicated to write, but classical and work with the methods of high-level languages such as C, and the other solution is "coroutines", which is a long forgotten art form, probably best described in Knuth's "The Art of Computer Programming", which is not supported by C, but requires a small assembly stub. It is (or was) supported by BCPL, as implemented in Tripos, and probably one of the reasons why the OFS used it. FFS uses its own coroutine functions, and so does CrossDos and CDFileSystem.

There is more to tell with CrossDos. My experience with it is that most people assume that it works like FAT95, but it does not. FAT95 is not multithreaded, and it is not quite as capable as CrossDos. Again, we need to take a detour for the details. On the Amiga, partitions are indicated by the Rigid Disk Block, a couple of blocks at the start of a disk that tells where the partitions start and end. This rigid disk block is to be interpreted by the firmware of the host adapter. This is for example the Guru ROM, or the GVP boot rom, or even part of the scsi.device. It is not the FFS or OFS that deals with it. Floppies don't use a Rigid disk block, and removable media such as ZIP should not use it either (this is a bad idea, more later).

Of course, PCs don't use a rigid disk block. They use a Bios MBR partiton table (not nowadays anymore with UEFI, they use yet another structure). The problem is: The hostadapter cannot read this structure (there is not even a hostadapter there), so nobody in the Os feels responsible for the MBR partition table. So CrossDos has to jump in. It can be run in two modes:

- Using the drive entirely as if it would be a floppy (this is the "superfloppy" mode)
- Interpreting the partition table, then handling one partition of the drive. It then needs to know which partition it should be responsible for.

Back when CrossDos became implemented, the new "SuperFloppy" flag in the mount list did not exist, so they looked for other solutions, which is the DosType field in the mount list.

So, if the dos type is MSD\0, it is a superfloppy, no partitions are assumed, and the whole disk is just one file system. This is identical to setting "SuperFloppy = 1" in the mount list. There is a second equivialent dos type, namely MDD\0 which does right the same. This is probably some legacy mechanism to switch between single and double density, but it is no longer in use and the two are now equivalent. One single file system per device.

If the DosType is MSH\0, instead a MBR partition table is assumed to be on the drive, and then "SuperFloppy" must be 0, i.e. off. Of course, then CrossDos needs to know which partition to access, and there is no such field in the mount list. Instead, it uses the name of the handler. If the device name ends with C, it is the first partition, if it ends with D, it is the second partition and so on. This is probably to mirror the typical Dos "partition names" where C: is the first partition on the internal hard disk, D: the second and so on. Hence, you cannot name the devices as you like, but the name of the icon in DEVS:DOSDrivers nees to end with the right character names.

Thus, to keep it short, three things need to be fixed to switch over from FAT95 to CrossDos:
a) Change the FileSystem entry in the MountList to L:CrossDosFileSystem
b) Change the dos type to  0x4d534800 for MBR-partitioned drives or 0x4d534400 for super floppies (or keep 0x4d534800 and add "SuperFloppy = 1").
c) Change the name of the icon such that the last letter indicates the right partition.

FAT95 uses a different dos type, namely 0x46415400, and uses a different way to indicate partitions, and people often forget to change this. In particular, FAT95 uses the last digit of the Dos type to indicate the partition number, i.e. 0x46415401 is the next partition.

There is, however, something wrong with this hack, and it is a bad idea what FAT95 attempts here. And for this, we need another excursion, and this is the FileSystem.resource. This is an invention that came into live with v34 (Kick 1.3) and autobooting. The problem it tries to solve is that if you have a harddisk with many partitions and a floppy, then each of these devices needs a file system (clearly). However, without any further trick, the same file system needs to be loaded once per partition, hence for a three-partition drive and two floppies, five copies of the FFS would have to be loaded into RAM. This is clearly a very bad idea and wasteful.

Instead, we have the FileSystem.resource, which "buffers" file systems. So, whenever a partition is mounted, the Mount command (or the firmware of the host adapter) checks in this resource for a file system with the same DosType as the one it needs for mounting the partition, and if it finds one,  it just recycles what it finds there. Hence, there are three CrossDos entries in the FileSystem.resource once you mount a crossdoss device, installed there by CrossDos itself: MSH\0, MSD\0 and MDD\0, and all three point to the same CrossDos code.

Now, consider the same for FAT95: Instead, you could have as many as 255 entries in the FileSystem resource for that: One for FAT\0, the first partition, one for FAT\1, the second partition and so on, up to FAT\255, the 256th partition. This is clearly not practical. FAT95 mis-uses the DosType for something it was not designed for, and its authors probably forgot about the implications this has for the FileSystem.resource.

In the meantime, CrossDos had one bug fixed, namely that the version that came with 3.1.4 accepted only FATs (directories) with at most 16 blocks in size. This was part of a legacy check mechanism to identify corrupt disks, but is no longer relevant for today, so it got removed. CrossDos now *also* supports the FAT\xx dos type just because I know that only a minority of users read instructions, or even this text, but there is no entry (for reasons given above) for this DosType in the FileSystem.resource, so the code isn't shared. It just loads CrossDos again from disk, wasting your memory.

Avoid the waste, change the DosType.
 
The following users thanked this post: Steady

Offline NinjaCyborg

Re: Os 3.1.4 - List of bug fixes and changes by component
« Reply #102 on: December 09, 2018, 11:35:30 AM »
Great stuff thanks Thomas. So for a USB memory stick or memory card formatted FAT for portability, and assuming you have a USB stack, what's the best way to treat that like a superfloppy without creating individual mountlists for every stick you have?
 

guest11527

  • Guest
Re: Os 3.1.4 - List of bug fixes and changes by component
« Reply #103 on: December 09, 2018, 12:20:45 PM »
Great stuff thanks Thomas. So for a USB memory stick or memory card formatted FAT for portability, and assuming you have a USB stack, what's the best way to treat that like a superfloppy without creating individual mountlists for every stick you have?

Your average USB stick has a partition table on it, with one single partition, so an MSH type mount list with a device name ending with a 'C' should do the trick.
 

Offline NinjaCyborg

Re: Os 3.1.4 - List of bug fixes and changes by component
« Reply #104 from previous page: December 09, 2018, 12:49:12 PM »
But will that work if I unplug it, then change it for a different one? Or put two at the same time, in different USB sockets, both of which would therefore be "C" drive assuming one partition per stick?