Welcome, Guest. Please login or register.

Author Topic: Os 3.2 development preview  (Read 135024 times)

Description:

0 Members and 4 Guests are viewing this topic.

guest11527

  • Guest
Re: Os 3.2 development preview
« Reply #314 from previous page: November 29, 2019, 08:57:55 AM »
Next episode: trackdisk.device.

Actually, one probably wouldn't expect much news from such an old device, but yet there is. The trackdisk.device suffered from two bugs which remained undetected in 3.1.4.

One is that with a properly crafted sector layout, the trackdisk device will overwrite memory beyond the track buffer because it does not check the sector index fully.

Another is that the device uses a busy-loop to time the step motor frequency, though this loop is not throttled by any type of hardware register access. This means that on fast hardware, floppy seeking becomes unreliable because the stepping frequency becomes too high. Now the loop is throttled by a hardware register access, so the timing should be (more) reliable.
 
The following users thanked this post: Tygre

Offline kolla

Re: Os 3.2 development preview
« Reply #315 on: November 29, 2019, 09:46:27 AM »
Right, but why not just use a more available and less obscure hotkey as default in the first place.
Such as RETURN probably? Come think about it, would you?
I was not thinking of Enter, no. The otherwise useless caps-lock would be my choice.
B5D6A1D019D5D45BCC56F4782AC220D8B3E2A6CC
---
A3000/060CSPPC+CVPPC/128MB + 256MB BigRAM/Deneb USB
A4000/CS060/Mediator4000Di/Voodoo5/128MB
A1200/Blz1260/IndyAGA/192MB
A1200/Blz1260/64MB
A1200/Blz1230III/32MB
A1200/ACA1221
A600/V600v2/Subway USB
A600/Apollo630/32MB
A600/A6095
CD32/SX32/32MB/Plipbox
CD32/TF328
A500/V500v2
A500/MTec520
CDTV
MiSTer, MiST, FleaFPGAs and original Minimig
Peg1, SAM440 and Mac minis with MorphOS
 

guest11527

  • Guest
Re: Os 3.2 development preview
« Reply #316 on: November 30, 2019, 08:16:52 AM »
Another component: The con-handler.

The con-handler is providing the window the shell runs in, but also other applications can open a CON: and request text lines, or - as a variant - keyboard presses from RAW:, the "non-cooked" version of CON:

There is surprisingly much about CON:, its improvements and defects. Probably some bugs first:

*) ACTION_WAIT_CHAR aka WaitForChar() was buggy, if several tasks inititated ACTION_WAIT_FOR_CHAR simultaneously, the handler might have done strange things. It neither counted the number of lines correctly to make ARExx happy.
*) Errors returning from the output device, i.e. typically the console.device, were not forwarded to the caller.
*) The parser of the incoming CSI sequences from the console.device improved. Now cursor movements by more than one position are supported - and also necessary as part of ACTION_FORCE and the tab expansion. ESC[ was not handled as substitute for CSI, though it is necessary to do so for 7-bit connections (see also below).
*) ACTION_STACK and ACTION_QUEUE (both for ARexx) had some issues not collecting data correctly. ACTION_FORCE was implemented outright incorrectly and did not offload its data into the keyboard input buffer, but the output buffer.
*) CON: forgot to re-enable the close-window event when switching from RAW mode to CON mode, thus leaving the user sit with a function-less close gadget.
*) ACTION_DISK_INFO disabled the auto-ability of the window without a chance to re-enable it. CON: now implements ACTION_UNDISK_INFO, which is already in use by some programs (Ed, More, Dir and some more).
*) An Off-by-one error may have read uninitialized memory when pasting into the console.
*) Any pending read request was not answered when closing the window, potentially causing a crash.
*) The con window is now opened in "new style" colors. While it makes typically no difference, it implies that the "Ed" window is now black on white, not yellow on black.
*) ACTION_GET_VARS was never documented, and allowed access to the (ever changing) global variables of CON: This is an interface nightmare that was just dropped for sanity.
*) ACTION_REPLACE, ACTION_PEEK, ACTION_SIZE_HIST, ACTION_SET_HIST and ACTION_GET_HIST were documented, but never implemented. As their purpose was unclear, their support was dropped.
*) Actually, the entire history functionality was dropped. This is done by the shell in a much better way with a much saner interface.
*) The console supports now ViNCEd's "medium mode" which forwards history requests and TAB expansions as special CSI sequences to the shell. Which makes use of it. So we finally have TAB expansion in the console, by the right party, which is the shell.
*) The con-handler had a very strange bug where, if you left the cursor sitting at the start of a physical line that is part of a long logical line going through more than one physical line, the console.device broke the line implicitly. Thus, when resizing the window, the cursor was not positioned correctly and the logical line was split. This was very hairy to debug and to fix as it depends on some pecularities of the console.device.
*) If concurrent input and output was happening, results may have been strange as the con-handler attempts to buffer them, and delay its execution, though not always with the correct end result as input and output cursor/replace/insertion activities are not commutatitive and cannot be interchanged freely.
*) The con-handler was extended not only to operate on top of the console.device, but any other device as well. As such, it replaces now the AUX-Handler completely. In fact, all of AUX what is left is a small stub function that initiates the con-handler with the right parameters. Which implies that you can get TAB-Expansion on a shell running over a serial line. Yay!
*) A forward slash in the window title can now be escaped by a backwards slash, and the backwards slash by a backwards slash itself. This also allows complex window titles.
*) The console window can now also optionally be equipped with an iconification gadget, all provided conclip is running to take care of the events. This allows iconification of shell windows.
*) Also, the console window allows dropping icons into it which are then expanded to the corresponding file names as if typed on the keyboard, including escaping of special characters. Again, this requires the help of conclip.

So, we are moving towards a much smarter console. Unfortunately, there is not yet a scroll gadget because this would require support from the console.device - which I'm currently not trying to touch. But this will come, sooner or later.




 
The following users thanked this post: Tygre

Offline kolla

Re: Os 3.2 development preview
« Reply #317 on: November 30, 2019, 03:45:02 PM »
What if console-handler had built in multiplexer with scrollback buffers...
B5D6A1D019D5D45BCC56F4782AC220D8B3E2A6CC
---
A3000/060CSPPC+CVPPC/128MB + 256MB BigRAM/Deneb USB
A4000/CS060/Mediator4000Di/Voodoo5/128MB
A1200/Blz1260/IndyAGA/192MB
A1200/Blz1260/64MB
A1200/Blz1230III/32MB
A1200/ACA1221
A600/V600v2/Subway USB
A600/Apollo630/32MB
A600/A6095
CD32/SX32/32MB/Plipbox
CD32/TF328
A500/V500v2
A500/MTec520
CDTV
MiSTer, MiST, FleaFPGAs and original Minimig
Peg1, SAM440 and Mac minis with MorphOS
 

Offline Matt_H

Re: Os 3.2 development preview
« Reply #318 on: November 30, 2019, 04:38:04 PM »
Quote
So, we are moving towards a much smarter console. Unfortunately, there is not yet a scroll gadget because this would require support from the console.device - which I'm currently not trying to touch. But this will come, sooner or later.

I'm genuinely curious: WShell, KShell (and others? ViNCEd?) have had scroll gadgets for a long time. How do they implement them? Hacks?
 

guest11527

  • Guest
Re: Os 3.2 development preview
« Reply #319 on: November 30, 2019, 05:03:37 PM »
I'm genuinely curious: WShell, KShell (and others? ViNCEd?) have had scroll gadgets for a long time. How do they implement them? Hacks?
I do not know how the former two work, but ViNCEd is not based on the console.device at all. All its console handling is entirely on its own, which is also the reason why the latter implements a more or less complete VT220 CSI command set, and the console.device and CON: only a subset of it.

It will require some research on my side whether we can get away with some smaller modifications of the console.device or need a rewrite of it, but this is just another dragon I do not quite want to attack right now as there is a lot more to do.

The console.device is just another of these obscure arcane parts of the Os. Entirely assembler, hard to read, hard to update. It will take some time.
 

Offline Matt_H

Re: Os 3.2 development preview
« Reply #320 on: December 01, 2019, 02:13:15 AM »
KShell adds an iconify gadget to the window as well, so I wonder if there are some hooks into Intuition?

I just realized the OS4 shell/console also has a scroll gadget, so maybe there's some documentation/code you could borrow from that when the time comes.
 

guest11527

  • Guest
Re: Os 3.2 development preview
« Reply #321 on: December 01, 2019, 06:57:56 AM »
KShell adds an iconify gadget to the window as well, so I wonder if there are some hooks into Intuition?
I will report about intuition of course. Yes, there is a new system gadget, and you can request it with an appropriate flag, and it sends an IDCMP like every other system gadget.

I just realized the OS4 shell/console also has a scroll gadget, so maybe there's some documentation/code you could borrow from that when the time comes.
Not for 3.2, thank you. There is more than enough to do.
 

guest11527

  • Guest
Re: Os 3.2 development preview
« Reply #322 on: December 01, 2019, 09:18:09 AM »
Another new component: The ram-handler

The most important new feature of the v47 ram-handler are "external links", i.e. links from the ram disk to outside file systems that are resolved as soon as a file or directory is accessed. You would typically use this to point "RAM:ENV" to "ENVARC:" such that any file that is accessed in ENV: is automatically copied from ENVARC: to ENV: when needed. There is no need for HappyEnv or any other dedicated env-handler as RAM: does it all itself.

There are some improvements in memory management, though. Regularly sized blocks (i.e. full size) are taken directly from exec instead from the pool as there is no benefit going through the indirection. RAM is, however, now smarter saving space by only allocating what is needed for the file. Whereas the previous versions of RAM: allocated always entire blocks, except for very small files, the v47 version ensures that the last block of a file does not take an entire block, but only the fraction that is needed.

Finally, we had one interesting new/old bug, in (sigh) ExNext() (how unsurprising). If you deleted a file pointed to by fib_Key in the FileInfoBlock, and the memory taken for this file was overwritten or released, ExNext() on the (then stale) key could have caused a MuGA hit. The hit itself was harmless, but the code that caused the hit was non-sensical, so was repaired and fixed. That was not a new bug, but an old one that might have been there all the way from the ancient days.

This said, did I mention that ExNext() is misdesigned?


 
The following users thanked this post: Tygre

Offline kolla

Re: Os 3.2 development preview
« Reply #323 on: December 01, 2019, 10:17:19 AM »
You would typically use this to point "RAM:ENV" to "ENVARC:" such that any file that is accessed in ENV: is automatically copied from ENVARC: to ENV: when needed.
The only time a copy in env: is needed, is when the content differs from that in envarc:, so instead of copy on access, it would make sense to just copy on write to env: and on access follow the link to envarc:.
B5D6A1D019D5D45BCC56F4782AC220D8B3E2A6CC
---
A3000/060CSPPC+CVPPC/128MB + 256MB BigRAM/Deneb USB
A4000/CS060/Mediator4000Di/Voodoo5/128MB
A1200/Blz1260/IndyAGA/192MB
A1200/Blz1260/64MB
A1200/Blz1230III/32MB
A1200/ACA1221
A600/V600v2/Subway USB
A600/Apollo630/32MB
A600/A6095
CD32/SX32/32MB/Plipbox
CD32/TF328
A500/V500v2
A500/MTec520
CDTV
MiSTer, MiST, FleaFPGAs and original Minimig
Peg1, SAM440 and Mac minis with MorphOS
 

guest11527

  • Guest
Re: Os 3.2 development preview
« Reply #324 on: December 01, 2019, 10:31:20 AM »
The only time a copy in env: is needed, is when the content differs from that in envarc:, so instead of copy on access, it would make sense to just copy on write to env: and on access follow the link to envarc:.
That would conflict with how ENV: is used. A write into ENVARC: shall *not* be reflected in ENV: ENVARC: contains the "saved preferences". "ENV:" contains the "used preferences". The HappyEnv "copy on access" functionality proved useful and robust over the years. If a concept works, I'll take it.
 

Offline kolla

Re: Os 3.2 development preview
« Reply #325 on: December 01, 2019, 01:09:25 PM »
An exclusive write to envarc: is also reflected in env: today with happyenv/env-handler, but no program really does this.

I cannot think of any program that only writes to envarc: without also writing to env: - can you? “Save” means “save and use” for all prefs programs at least. Or do you plan to change that?
« Last Edit: December 01, 2019, 01:11:28 PM by kolla »
B5D6A1D019D5D45BCC56F4782AC220D8B3E2A6CC
---
A3000/060CSPPC+CVPPC/128MB + 256MB BigRAM/Deneb USB
A4000/CS060/Mediator4000Di/Voodoo5/128MB
A1200/Blz1260/IndyAGA/192MB
A1200/Blz1260/64MB
A1200/Blz1230III/32MB
A1200/ACA1221
A600/V600v2/Subway USB
A600/Apollo630/32MB
A600/A6095
CD32/SX32/32MB/Plipbox
CD32/TF328
A500/V500v2
A500/MTec520
CDTV
MiSTer, MiST, FleaFPGAs and original Minimig
Peg1, SAM440 and Mac minis with MorphOS
 

guest11527

  • Guest
Re: Os 3.2 development preview
« Reply #326 on: December 01, 2019, 03:08:24 PM »
I cannot think of any program that only writes to envarc: without also writing to env: - can you?
Yes.

“Save” means “save and use” for all prefs programs at least.
Turboprint.
 

Offline kolla

Re: Os 3.2 development preview
« Reply #327 on: December 01, 2019, 03:44:09 PM »
Really, sounds confusing - “save, but do not use!”

Well, then perhaps SetEnv should be improved with this feature too, “SetEnv SAVEONLY” or thereabouts... and likewise for OS prefs programs.
B5D6A1D019D5D45BCC56F4782AC220D8B3E2A6CC
---
A3000/060CSPPC+CVPPC/128MB + 256MB BigRAM/Deneb USB
A4000/CS060/Mediator4000Di/Voodoo5/128MB
A1200/Blz1260/IndyAGA/192MB
A1200/Blz1260/64MB
A1200/Blz1230III/32MB
A1200/ACA1221
A600/V600v2/Subway USB
A600/Apollo630/32MB
A600/A6095
CD32/SX32/32MB/Plipbox
CD32/TF328
A500/V500v2
A500/MTec520
CDTV
MiSTer, MiST, FleaFPGAs and original Minimig
Peg1, SAM440 and Mac minis with MorphOS
 

guest11527

  • Guest
Re: Os 3.2 development preview
« Reply #328 on: December 02, 2019, 06:58:26 AM »
Next component: The scsi.device.

This device is one of the "bad surprises" of the operating system over and over again. It started as an SCSI device for the A3000, and then was drilled up to support IDE, then again ATAPI, so the code quality is... quite moderate, to put it this way. It consists of many files that are conditionally used depending on the compiler option and the build target, and it is not easy to read and follow.

"Of course" this caused problems during development, and the infamous "maxtransfer" bug where data losses were caused if more than 255 sectors are transported in one go was, actually, in multiple places, not just one. One of them was a higher level bug we fixed, namely at the level where 'trackdisk-like' read and write commands are translated into 'scsi-like' commands. But what we forgot is that there is also *the very same bug again* on a lower level where the scsi-level commands are translated to IDE commands. Thus, if the command was not broken up into 255 sector pairs at 'trackdisk level', for example because the user selected "SCSI direct" options, then the same bug appeared again at the 'scsi to IDE' transfer level. Yuck.

So I hope we have get this bugger done for good.

There are a couple of other minor changes. For example 'TD_SEEK' was not implemented correctly. Actually, I wonder why anyone would want to issue this command - 'move the read/write head to the indicated sector' - on a harddisk. But, at least, the harddisk should not attempt to write to the surface if it is issued. This was an old bug that got fixed.

Then, the code used its own implementation for the "Gary IDE detection logic", but code for that is already present in the exec.library, and the code there also depends on the motherboard model by a compiler switch. Thus, instead of repeating the whole logic again in the scsi.device, the function now falls back to the exec.library.

Finally, some words of warning:

Concerning maxtransfer: While the scsi.device breaks now sector counts > 255 down into transfers of 255 bytes each, it *seems* that there are at least some buggy implementations of IDE around - typical flash chips - that consider the byte count as "signed byte" due to an oversight, and that do stupid things for counts > 127. So for *some arrangements*, users still have to use a maxtransfer value, though then of 0x0000ffe0. This is generally not needed, but only for buggy drives beyond our control.

And another warning: Some people still use IDEFix. First, this is unnecessary as the scsi.device implements ATAPI now fine, so a second level patch is not necessary. Second, IDEfix does not issue LBA48 commands as the scsi.device does, and hence limits the maximum drive capacity to (IIRC) 128MB. Any larger drive will appear in the HDToolBox only as 128MB drive. Then people come along, partition their drive/card under UAE, and insert it into the Amiga, and then all hell breaks loose because IDEfix cannot address any blocks beyond the 128MB limit, even though the partition appears to be the right size.

Solution is ideally not to use idefix, or simply "partition the drive on the system you want to use it on" because you then become aware of the limits of the software you depend on. Well, hopefully...

 
The following users thanked this post: Tygre

Offline kolla

Re: Os 3.2 development preview
« Reply #329 on: December 02, 2019, 07:18:08 AM »
So this is all bogus?

http://aminet.net/package/driver/media/IDEfix97

Quote
IDE-fix:
Enhance your IDE-port! Faster transfer, faster booting, less CPU use!
No MaxTransfer troubles, set MaxTransfer as big as you like!
TD64 & NSD commands supported, use IDE drives bigger than 4 GIG with your
Amiga (requires FileSystem with TD64 and/or NSD support).
Patches into the system "on the fly", no reboot required.
Use 4 IDE drives with your Amiga (with additional 4 drive adapter)!!!
Supports removable IDE units (SyQuest or ZIP IDE drives) without trouble!
B5D6A1D019D5D45BCC56F4782AC220D8B3E2A6CC
---
A3000/060CSPPC+CVPPC/128MB + 256MB BigRAM/Deneb USB
A4000/CS060/Mediator4000Di/Voodoo5/128MB
A1200/Blz1260/IndyAGA/192MB
A1200/Blz1260/64MB
A1200/Blz1230III/32MB
A1200/ACA1221
A600/V600v2/Subway USB
A600/Apollo630/32MB
A600/A6095
CD32/SX32/32MB/Plipbox
CD32/TF328
A500/V500v2
A500/MTec520
CDTV
MiSTer, MiST, FleaFPGAs and original Minimig
Peg1, SAM440 and Mac minis with MorphOS