Welcome, Guest. Please login or register.

Author Topic: THOR's Shell Hacks  (Read 35173 times)

Description:

0 Members and 2 Guests are viewing this topic.

guest11527

  • Guest
Re: Variable Shell Hacks
« Reply #44 on: February 11, 2015, 12:51:32 PM »
How much do you know about shell variables? These work like in any other programming language, REXX for example. You assign them a value like this

Code: [Select]
7.SCSI:> set foo bar

and you retrieve their value a little bit different, namely by putting a dollar sign in front of their name:
Code: [Select]
7.SCSI:> echo $foo
bar

Where do these variables come from? Well, from several places. First, if you just use "set" as above, they come from a shell internal database. This is different for each shell, and hence, such variables are "local" to the shell you use them.

A second type of variables are "global environment variables". They are set with "setenv" instead of "set", but work otherwise the same way:

Code: [Select]
7.SCSI:> setenv hi bar

and you retrieve their value a little bit different, namely by putting a dollar sign in front of their name:

Code: [Select]
7.SCSI:> echo $hi
bar

If two variables of the same name exists, i.e. one environment variable and a local shell variable, the latter is used. Actually, environment variables exist as a file in ENV:, an assign to a directory in the RAM-disk, which is usually copied from ENVARC: during startup. Thus, you can make environment variables permanent by copying their file from ENV: to ENVARC:

Or you can do that in one step:

Code: [Select]
7.SCSI:> setenv save hi bar

the "save" keyword also makes the change permanent and places the variable file in ENVARC: and in ENV:, so it will survive a reboot.

Now, there is a little bit more that can be said about variables? What do we do if we want to write "$foo" on the console, as a literal string, and do not mean the variable? Well, use the BCPL escape-character, which is the asterisk:

Code: [Select]
7.SCSI:> echo "The value of *$foo is $foo"
The value of $foo is bar

Now, here comes the wierd part: Didn't I say that the "*" works only as escape character within quotes? Yes, but... for *$ this is different, it works always:

Code: [Select]
7.SCSI:> echo *$foo
$foo

Why this inconsistency? Well, the original BPCL shell was drilled up over and over again, and apparently, CBM didn't quite know what they were doing, so consistency was apparently not an option, or the inconsistency was not recognized. In the old shell code, parsing and expansion of variables and parsing of strings and quotes was handled in two separate steps, so the code did not even know whether the $ was inside or outside of a double quote.

Speaking of double quotes, did you know that a double quote *within* a string is not understood as a double quote?

Code: [Select]
7.SCSI:> echo thisisa"doublequote
thisisa"doublequote

If there would have been a space in front of the quote, it would have acted as a quote, but this one did not. Every sane programming language would have created an "unterminated string" error for the above line. AmigaDos does not. A quote in the middle of a string is not a quote. Why? Nobody knows...

Back to variables: If you want other characters such as spaces in the variable name, or want to make sure that the variable name is separated from the string, you may enclude the name in curly braces:
Code: [Select]
7.SCSI:> echo ${foo}bar
barbar
without the curly braces, the shell would have tried to expand the variable foobar, which does not exist. If a variable does not exist, it just expands to its name, so let's try this:
Code: [Select]
7.SCSI:> echo $foobar
$foobar
Now, how can we know whether a variable exists? That's what $? is good for:
Code: [Select]
7.SCSI:> echo $?foo
1
7.SCSI:> echo $?foobar
0
7.SCSI:> echo $?{foo}bar
1bar
So $?var expands to 1 if the variable exists, and 0 otherwise, and the last example just showed that curly braces work with $? as well, to separate the variable name "foo" from the rest of the string "bar".

Tomorrow: More on variables. Or actually, things that look like variables, but really aren't. The "alias" command introduces something that works like a variable, but isn't. Again, only CBM knowns why they made this so inconsistent. And then, scripts can take arguments, filled in by "execute". These also look like variables, but strangely enough, they aren't. What CBM was smoking at this time, nobody knows, but it's really another completely different mechanism.
 

Offline kolla

Re: Variable Shell Hacks
« Reply #45 on: February 11, 2015, 03:33:25 PM »
I recall feeding ssh with password using + many many years ago, something like...

Quote
RequestString password
ssh myhost "remote commando" +
$password

Or on second thought, maybe I ended up using pipe because + did not work with ssh?
Anyhow I felt very dirty, hehe.
B5D6A1D019D5D45BCC56F4782AC220D8B3E2A6CC
---
A3000/060CSPPC+CVPPC/128MB + 256MB BigRAM/Deneb USB
A4000/CS060/Mediator4000Di/Voodoo5/128MB
A1200/Blz1260/IndyAGA/192MB
A1200/Blz1260/64MB
A1200/Blz1230III/32MB
A1200/ACA1221
A600/V600v2/Subway USB
A600/Apollo630/32MB
A600/A6095
CD32/SX32/32MB/Plipbox
CD32/TF328
A500/V500v2
A500/MTec520
CDTV
MiSTer, MiST, FleaFPGAs and original Minimig
Peg1, SAM440 and Mac minis with MorphOS
 

Offline kolla

Re: Variable Shell Hacks
« Reply #46 on: February 11, 2015, 03:38:33 PM »
Regarding curly braces - are they default for .bra and .ket now?
B5D6A1D019D5D45BCC56F4782AC220D8B3E2A6CC
---
A3000/060CSPPC+CVPPC/128MB + 256MB BigRAM/Deneb USB
A4000/CS060/Mediator4000Di/Voodoo5/128MB
A1200/Blz1260/IndyAGA/192MB
A1200/Blz1260/64MB
A1200/Blz1230III/32MB
A1200/ACA1221
A600/V600v2/Subway USB
A600/Apollo630/32MB
A600/A6095
CD32/SX32/32MB/Plipbox
CD32/TF328
A500/V500v2
A500/MTec520
CDTV
MiSTer, MiST, FleaFPGAs and original Minimig
Peg1, SAM440 and Mac minis with MorphOS
 

guest11527

  • Guest
Re: Variable Shell Hacks
« Reply #47 on: February 12, 2015, 06:07:06 AM »
Quote from: kolla;783636
Regarding curly braces - are they default for .bra and .ket now?

No. ".bra" and ".ket" have nothing to do with curly braces. That's something for today's episode. Confusingly, this is in no relation to shell variables.
 

Offline pVC

Re: Variable Shell Hacks
« Reply #48 on: February 12, 2015, 09:26:39 AM »
BTW. any idea how can I have parenthesis in destination name/path with Copy command? Using ' characters before parenthesis do work in source, but not in destination. At least in standard shell, haven't tried with Vinced...

For example:

Ram Disk:test> echo "bla" > test(1)
Ram Disk:test> dir
  test(1)
Ram Disk:test> copy test'(1') testfile
Ram Disk:test> dir
  test(1)                          testfile
Ram Disk:test> copy testfile test(2)
Wildcard destination invalid.
Ram Disk:test> copy testfile test'(2')
Wildcard destination invalid.
Ram Disk:test> makedir testdir(1)
Ram Disk:test> dir
     testdir(1) (dir)
  test(1)                          testfile
Ram Disk:test> copy testfile testdir(1)/
Wildcard destination invalid.
Ram Disk:test> copy testfile testdir'(1')
Wildcard destination invalid.
« Last Edit: February 12, 2015, 09:30:56 AM by pVC »
Daily MorphOS user and Amiga active.
 

guest11527

  • Guest
Re: Variable Shell Hacks
« Reply #49 on: February 12, 2015, 12:50:26 PM »
Quote from: pVC;783722
BTW. any idea how can I have parenthesis in destination name/path with Copy command?

You can't. )-: It's not even related to the shell, or the console (i.e. ViNCEd). The trouble is the following: The Os pattern matcher pre-parses the pattern to get a more efficient internal representation to do the matching algorithm. That's actually part of the pattern matcher CBM copied/found in arp (the AmigaDos replacement project). Unfortunately, while this pattern matcher correctly implements the apostrophe as escape character (why not the star, it already is the escape character? Who knows...), it unfortunately still tells you "ok, there was a pattern here. A pattern I ignored, though still...". And then copy fails. I'll call this a bug.
 

Offline ChaosLord

  • Hero Member
  • *****
  • Join Date: Nov 2003
  • Posts: 2608
    • Show only replies by ChaosLord
    • http://totalchaoseng.dbv.pl/news.php
Re: THORs Shell Hacks
« Reply #50 on: February 14, 2015, 09:22:22 PM »
I just noticed that AmigaKit advertises AmigaOS4.1 Final Edition for Classic Amigas with this line:

Quote from: AmigaKit Website

Most powerful console for AmigaOS


Does this mean that OS4.1 uses the THOR-console?
Wanna try a wonderfull strategy game with lots of handdrawn anims,
Magic Spells and Monsters, Incredible playability and lastability,
English speech, etc. Total Chaos AGA
 

guest11527

  • Guest
Re: THORs Shell Hacks
« Reply #51 on: February 15, 2015, 10:10:25 AM »
Quote from: ChaosLord;784139
Does this mean that OS4.1 uses the THOR-console?

Certainly not. ViNCEd is written in assembly language, and that prevents porting.
 

guest11527

  • Guest
Re: THOR's Shell Hacks: Execution and alias
« Reply #52 on: February 15, 2015, 02:37:43 PM »
Ok, let's look at today's episode of the shell hacks. We already had variables and how to use them in scripts, but there are more places where elements of the shell are substituted that look like variables - but are not.  First, there is the alias command, a shell-built-in:

Code: [Select]
7.SCSI:> alias ls list


This creates an alias, also "a new name for a command". Now, whenever you type in "ls", the shell substitutes that for you by "list". Where are aliases actually stored? Well, in the same list as local variables, so there have pretty much the same status: They are local to the current shell. Thus, if you want to establish an alias for all shells, you need to put this into S:Shell-Startup where the shell picks it up every time you start one from the workbench.

Aliases also may take arguments. Or rather, a single argument. Without any further preparation, the argument to the alias are simply appended to the contents of the alias, i.e.
Code: [Select]
7.SCSI:> ls df0:
expands to "list df0:" because the alias is extended. If you want to put the arguments of the alias elsewhere, you need to tell the shell where it should put it by using a pair of square brackets. (Why not ${arg} or $* or something? Oh, CBM, consistency, consistency!)

For example:
Code: [Select]
7.SCSI:> alias xd type [] hex
would create a new alias "xd" (probably short for "hex dump") that, when expanded, puts its arguments between "type" and "hex". Hence:
Code: [Select]
7.SCSI:> xd C:Copy
[code]
expands to "type C:Copy hex", and prints out a hex dump of the copy command in C:. Thus, [] is some sort of a variable, but not really, as it only goes for alias and nowhere else.

Can aliases be recursive? Well, partially:
[code]
7.SCSI:> xd ls
expands to (expectedly) "type list hex", because it expands the "xd" alias and the "ls" alias. Unless you are in C: or have a "list" in the current directory, this will of course create an error.

The following, however, does not work:
Code: [Select]
7.SCSI:> xd xd
which will expand to "type xd". The rule is simple: In a single argument line, a given alias can be expanded at most once. This is simply because the shell could otherwise run into an infinite recursion, trying to expand the same alias again and again.

Then, what do you do if you want a literal [ somewhere in an alias? You guess right, put an asterisk upfront. That's the BCPL escape character, and like with variables, it also works here outside of quotes and not only within quotes.

Then, we have a third citizen that has something "almost like but not quite" variables. It is execute, and shell scripts. You can define arguments to shell scripts by defining an "argument template" in the first line of a script:

Code: [Select]
.key filename
type hex

This is a simple script that takes a single argument, denoted "keyname", and then prints the contents of the file in hex, pretty much like "xd" did. Now, what do the angle-brackets do here?

This is, confusingly, not command redirection, but "argument substitution". Again, an inconsistency. "type ${filename} hex" would have been much nicer.

At this point, the shell is completely innocent. The arguments are not even touched or ever seen by the shell. What happens here is all the magic of the "execute" command: It parses the input script itself before forwarding it to the shell, and defines its *own* mechanism for substituting arguments, and does not even use or look at shell variables. In fact, the $ (dollar) sign the regular shell uses to indicate variables has a completely different meaning within execute scripts. It indicates a default value in case an argument is not defined, i.e

Code: [Select]
type hex

uses "C:Copy" as default if "filename" is left blank. Did I say that CBM was highly inconsistent? Did I say that this means that input/output redirection in execute-scripts is hard because angle brackets have another meaning? Probably the author of execute should be ... executed?

Ok, at least one can do something, namely define other characters for such functions:
Code: [Select]
.key filename
.bra {
.ket }
.dollar |
type {filename|C:Copy} hex
which looks much nicer. So you now have "{" and "}" as argument separator, and the vertical bar as default indicator, though that means that you need to spell variables without the curly brackets, otherwise you get nonsense:

Code: [Select]
key filename
.bra {
.ket }
.dollar |
echo "your kistkart version is ${kickstart}"
type {filename|C:Copy} hex
will not do what it should. Yuck!

Dear CBM engineers, this is right away nonsense. The next time you attempt to "design" something, think twice. What about simply putting arguments to scripts into regular variables, and simply use the variables for passing them into commands?

Ok, to be fair, that's not quite how it worked. Execute came first, and it was extended from pure script execution to argument expansion, then came shell variables. Thus, CBM should probably have made variables consistent with execute arguments, which would have meant to loose input/output redirection. No, wait, that's not an alternative either...

One way or another, somebody screwed "execute" and there are only limited means to work around it. Only some parts can be un-screwed by the shell, see the "Skip Lab" episode. Execute is not actually "executing" something. It first runs a very simple substitution algorithm, puts the result of the substitution into T:, and then runs the shell on the parsed result. Instead of manually parsing the arguments, it would have been nicer to leave that to the shell, but then...

While we are at it: How do the ".bra" ".ket" ".key" and ".dollar" commands actually work? Actually, they don't. They don't do anything. If you look at the code for these commands in the shell source, they don't do anything. Instead, they just feed the argument parser of "Execute", so "execute" reads them, and does as indicated, but not the shell. If the shell finds a ".bra", it shrucks with its shoulders and says "ok, good for you, let's ignore this stuff..."
 

Offline ChaosLord

  • Hero Member
  • *****
  • Join Date: Nov 2003
  • Posts: 2608
    • Show only replies by ChaosLord
    • http://totalchaoseng.dbv.pl/news.php
Re: THOR's Shell Hacks: Execution and alias
« Reply #53 on: February 15, 2015, 03:10:42 PM »
Do you happen to remember what version of AmigaOS first introduced the "Parameter Passing via [] feature of the Alias command" ?

p.s. You have completely melted my brain. :crazy:
Wanna try a wonderfull strategy game with lots of handdrawn anims,
Magic Spells and Monsters, Incredible playability and lastability,
English speech, etc. Total Chaos AGA
 

guest11527

  • Guest
Re: THOR's Shell Hacks: Execution and alias
« Reply #54 on: February 15, 2015, 05:13:52 PM »
Quote from: ChaosLord;784249
Do you happen to remember what version of AmigaOS first introduced the "Parameter Passing via [] feature of the Alias command" ?
Not exactly. I would guess it must have come in in v37, around the same time the BCPL shell of v34 was replaced by the C shell. V34 does not have it, though it includes the alias list.
 

Offline kolla

Re: THOR's Shell Hacks: Execution and alias
« Reply #55 on: February 15, 2015, 05:38:14 PM »
If I remember correctly one can also use .bra and .ket many times and write very messy and unreadable scripts :)
B5D6A1D019D5D45BCC56F4782AC220D8B3E2A6CC
---
A3000/060CSPPC+CVPPC/128MB + 256MB BigRAM/Deneb USB
A4000/CS060/Mediator4000Di/Voodoo5/128MB
A1200/Blz1260/IndyAGA/192MB
A1200/Blz1260/64MB
A1200/Blz1230III/32MB
A1200/ACA1221
A600/V600v2/Subway USB
A600/Apollo630/32MB
A600/A6095
CD32/SX32/32MB/Plipbox
CD32/TF328
A500/V500v2
A500/MTec520
CDTV
MiSTer, MiST, FleaFPGAs and original Minimig
Peg1, SAM440 and Mac minis with MorphOS
 

guest11527

  • Guest
THOR's Shell Hacks: Have you sparwed today? Or h is not for hidden.
« Reply #56 on: February 17, 2015, 03:58:46 PM »
No, that's not a commercial for a new food fetish. It's on the Amiga protection bits, or at least the protection bits how the (not-so) FastFileSystem defines it and the shell uses it.

Protection bits can be added or removed by the "protect" command, such as
Code: [Select]
7.SCSI:> protect ram:test s add
Sets the "s" protection bit. If you replace "add" by "remove", the bit is removed again.

In total, we have eight of them: hsparwed. The protection bits are partially interpreted by the shell, and partially by the filing system. Now, for the meaning, one by another:

If "d" is set, the file can be deleted, or "unlinked". If it is not set, "delete" on the file fails. This also implies that the file cannot be opened for overwriting, i.e. MODE_NEWFILE for Open fails if the bit is cleared. What it means for directories is pretty much up to the filing system. The current FFS interprets "d" as "the directory cannot be removed", but you can create files in it.

Before we go to "e", first "w": If it is not set, it means that the file cannot be written to. Or rather, almost. You can append to the file, but you cannot overwrite it, at least this is how the FFS currently interprets this, and I would probably call this a bug, because it means that you *can* write to a file, even though w is not set. The RAM: disk has the same issue. Interestingly, this probably changed with the FFS when it was re-implemented in assembler, from the BCPL OFS code.

For directories, "w" should probably imply that you cannot create files within. But it doesn't. Actually, I seem to remember early releases of the FFS did not store any flags for directories.

Regardless of "w", the files or directories can be deleted, so at least a bit of consistency.

"r" means that the file can be opened for reading. At least for files. For directories, one is tempted to say that it means that the directory contents cannot be read, but AmigaOs is not Linux, and consistency isn't its strength anyhow, so "r" doesn't mean anything for directories.

So does "e". Actually, "e" is not interpreted by the filing system at all. It is only interpreted by the shell. If the "e" bit is set, the shell considers a file a "binary executable", and loads it. Or rather, tries to. If that fails, an error is printed. Actually, if "e" and "s" are both set, then "s" takes priority and "e" is ignored.

Actually, the above is not entirely true: If "e" is set in a directory, then of course the shell does not try to execute the directory (how?). Instead, it ignores all other flags and implicitly changes the directory. Or rather, it loads the "CD" command from its built-in command list, and then passes execution to it.

The "rwed" flags are actually stored inverted in the file descriptor - if the bits are clear, they are printed as "set", and vice versa. Thus, if you don't do anything else with the file, the default is "rwed". Actually, not a very good default. "rwd" would have been better as it would have avoided that you try to execute your average text file. CBM at all its glory, once again.

So if what happens if "e" is not set? The shell checks whether the file can be opened by a data type, and if so, checkes whether the variable $VIEWER is set, and then passes control over to the command recorded in this variable. Thus, if $VIEWER is set, for example to "Multiview", you can view pictures, text, AmigaGuides... on the shell by simply typing their names. Ain't that cool?

The next bit is the "a" bit. It actually doesn't do much, except that it is cleared whenever the file is modified. Thus, one can set it by
Code: [Select]
7.SCSI:> protect foo a add
and whenever some program fiddles with "foo", the "a" bit goes away. Sounds useless? No, it isn't. It means that a backup program can skip over all those files whose "a" bit is set, as they haven't been touched since, and while it copies all programs back to another disk, it sets the "a" bit of all the files while it goes. Thus, it only copies what hasn't been modified - quite useful. Actually "SortCopy" (from Aminet) does just that.

The "s" is for "script". That is one of the flags only the shell cares about, and not the filing system. If it is set, the Shell tries to execute the file as a "script", regardless of the "e" bit. Thus, for that, it first checks the first two characters of the file. If they are "/*", the file is passed into AREXX. If they are ";!" or "#!", the script file is passed into the shell command that follows these two magic characters. That's what Linux does all the time, there "#!" is the magic "hash-bang". AmigaOs has it, too.

If neither "/*" nor "#!" nor ";!" is in the first line, then Amiga hands control over to "Execute". Which, as we know now, doesn't do anything special except messing with the file once again by substituting arguments in an completely inconsistent way, and then passing the result back into the shell for execution. Execute doesn't do the dirty job itself. It just creates the mess...

The "s" flag with all following flags are relatively new. The BCPL shell did not have them, and they came in relatively late.

"p" is for "pure". Which actually means not as much as one may think. It was an invention of the "AmigaDos Replacement Project" we've already heard about before: The idea was nice: Why do we have to load commands from the disk everytime again when we need them? So instead, if a command does not change its executable, its data section, and is re-entrant - can be called from itself and by several tasks at once - its code can simply be kept in memory, instead of requiring to load it from disk to memory every time again.

"ARP" had a mechanism for that: The file was checksummed, and if the checksum did not match, it was thrown out the next time. Of course, all that was too much for CBM, so the checksum went away. Instead, it is just a promise. If "p" is set, the command makes the promise "I hereby swear that I will not modify my data and by code. And if I do, I may crash next time". Good job, that's a great stable interface.

Hence, if "p" is set, you can load the command into the dos resident list (more on this later), and the Shell finds it there, and executes it from there rather than loading it from disk. And if "p" is not set, you can set it with "protect" nevertheless and try what happens if you make it resident... Errr, better don't. Only the program author can know that, and hence may set "p". And if it is not set, you better keep your hands off and don't make it resident.

This is how one can make the startup-sequence  a bit faster, namely by moving often-used commands resident:
Code: [Select]
resident C:assign
resident C:copy
So they are not loaded over and over again. And actually, there is an even better way how to do that, without "resident":

Which brings us to the last bit, often misunderstood: "h". Some programs read it as "hidden" and don't list directory entries that have this bit set. That's just wrong, folks. "h" is not for "hidden", "h" is for "hold". What that means is the following:

If a file has the "h", "p" and "e" bits set, so it promises to be pure, and can be executed, then the next time the shell loads the command, it makes it resident all by itself. Thus, the above lines can be avoided in the startup-sequence simply by setting the "h" bit for all those commands you want to keep in memory anyhow, and you planned to make resident, except that you don't need a resident command anymore.

Unfortunately, due to size constraints, this nice feature was "thrown out" of the V40 shell in last minute, so the Os 3.1 shell does not support it. The 3.9 version does, as there is no ROM size constraint, and "h" really stands for "hold".
 

guest11527

  • Guest
How would you like your console today? Rare, English, Medium or Well Done?
« Reply #57 on: February 20, 2015, 01:27:14 PM »
Today, I'd like to write about the console again, or what is known as CON:, RAW: or VNC:. Actually, you may know the two console handlers AmigaOs has to offer: CON: and RAW:.

For CON:, every line is buffered, the device sits and waits and gives the user the chance to edit the full line, until he presses return, and at that point, the entire line is transferred to the program that waits for input; usually the shell, you name it. This is also called the "cooked" mode, as user input is "well prepared until it's served", namely to the receiving program. That's a name that comes from Unix, and Amiga has picked that up.

Then, we have RAW: Unlike CON:, it does not provide any type of line buffering. Every single keystroke is transferred to the receiving program, and the keystroke is not even printed on the screen. Instead, the receiving program has to do that. Since the AmigaShell does not, RAW: is not useful for the shell. Some third party shells provided better editing facilities, and then did so by implementing all history and editing themselves, building upon RAW: rather than CON:, because CON:s editing services are (or were) rather limited. At least until ViNCEd, which offers a very rich editor and does a lot more than CON: for that matter.

What is probably less known is that CON: and RAW: are two "flavours" of the very same console handler. One can open a console handler in "raw" mode, and switch it to "cooked" mode by a specific command, and also go back. This "almost" works, for CON: at least. In a RAW window, a program can, for example, ask the console to send information whenever the user resizes the window. In cooked mode, this does not work. In raw mode, a program can ask the console to send a string when the user clicks somewhere. In cooked mode, this does not work.

Or at least, did not work. ViNCed, of course, is no difference, and can be both "raw" or "cooked" - or as it is called here "rare" or "well done", as I you don't "cook" steak. In ViNCEd, this stuff works right, and you receive such messages, no matter whether the console is "raw" or "cooked" - ehem - "rare" or "well done". If a program asks for information on window resizes, it gets them.

Of course, ViNCEd's "cooked", ehem, "well done" mode is really pretty rich: It not only offers line buffering, but full screen editing. It also does TAB expansion, and job handling. It keeps a history of commands. That's probably all stuff that should better go into the Shell, as it is the matter of the shell (and not the console) what is going on at the command line. It didn't, because the AmigaShell evolved slower than the console.

As I did not like this architecture either, and wanted to open the possibility to relocate tab expansion, history, and job control into the shell, ViNCEd has four modes instead of two: "rare", "english", "medium" and "well done".

"Medium" is slightly less cooked than "well done": The console still does the full sceen editing handling, but leaves TAB expansion, job handling and history to the shell. Thus, what happens here is that - instead of expanding file names with TAB - the console picks up the file name to be expanded from the console, places the file name into a control sequence, and forwards the request to the Shell. Same goes for job control and history. Thus, it is the shell that has to receive such control sequences and act accordingly.

Of course, the AmigaShell is currently unable to do that, so the "medium mode" remains currently unused.

Then, we have the "English" mode, which is slightly more cooked than "raw", ehem, "rare". This mode works very much like the raw mode, except that the "shell features" such as TAB expansion, history and job control get special control sequences, allowing the user to relocate such functions to other keys on the keyboard, and the shell (again) receives such keyboard sequences instead of the "pure raw" sequences. Thus, if the user presses TAB, a "raw" console would send the ASCII character for TAB (08). In the "english" mode, the console would rather send a control sequence indicating "Hey, the user wants to do TAB expansion", and if the user redefines the keyboard, the same function would be bound to - for example - F1, and TAB would continue to send the ASCII character for the tabulator.

Thus, "english mode" distinguishes between "keyboard functions" that can be re-bound to other keys, and "raw keyboard codes" that remain bound to the native keys, unless overridden by a function. Keyboard redefinition thus remains, in English mode, to the console, execution of such sequences to the shell.

Of course, this mode is also completely unused as of today. It is just there.

Now finally, how does the console actually distinguish all these modes? Essentially, it is in the mount list, i.e. the part in DEVS:DOSDrivers that defines how a handler is loaded. For the old CON: console, this file is in ROM (or rather, its binary equivalent). For ViNCEd, it is in DEVS:DosDrivers. For the "well done mode", this file looks as follows:

Code: [Select]
Handler     = LIBS:vnc.library
Priority    = 4
StackSize   = 4096
GlobVec     = -2

Interestingly, the handler does not sit in L:, but rather in LIBS: because it is *also* a library (a neat trick, there is only one file for both functions, which is entirely possible).

For a "raw" console (typically called VNR: instead of VNC: or NEWCON:), it reads as follows:
Code: [Select]
Handler     = LIBS:vnc.library
Priority    = 4
StackSize   = 4096
GlobVec     = -2
Startup     = 1

The difference is in the "startup" line. Startup = 0 (or no startup) is "well done", startup = 1 is  "raw".

Startups "2" and "3" are the two new hidden modes.

Actually, the same trick, namely to use startup to switch between various "flavours" is also used in different edge of the operating system. SER:, PAR: and PRT: are all implemented by the same handler, namely the Port-Handler. It sits in L: The only difference between the three flavours of it are the values of "Startup". Of course, you usually don't see that because there is no entry in the mount-list. Instead, there is a binary equivalent of this mount entry built into the ROM.

There is small program, though, which displays these mount settings. It is called "Devices" and can be found in Aminet. It shows you what the entries look like, even without an explicit mount list to look at.
 

Offline Boot_WB

  • Hero Member
  • *****
  • Join Date: Feb 2005
  • Posts: 1326
    • Show only replies by Boot_WB
    • http://www.hullchimneyservices.co.uk
Re: THOR's Skip Lab
« Reply #58 on: February 20, 2015, 02:20:04 PM »
Quote from: Thomas Richter;783069
Today, I'll try to cover a bit a dark edge of the Amiga Shell, and these are the two commands "SKIP" and "LAB", and why they sometimes didn't work before 3.9. Of the two, LAB is actually not a command at all - or rather,  it is, but it doesn't do anything, while still being necessary.

Flow control - that's a key feature of every programming language. That is, to control where program flow goes. "if-then", "for-next", "do-while", "repeat-until", this kind of stuff. Unfortunately, the only thing AmigaOs has is "if-then", and "goto", even though the latter is pronounced "skip". It's the same ugly command that has been banned from many more reasonable languages for good reason - it creates spagetti code.

Anyhow, it's all we got, so here is how it works: If you have a "lab foo" somewhere in a shell script, then a "skip foo" continues control to this very position in the script. Simple enough? Not quite. If the label is *above* the skip command, i.e. on a line "higher up in the code", you need to say "skip BACK foo".

Simple enough? Not quite. If there is an "execte" somewhere between the label, or you are calling a shell script by some other means - probably because you have a file whose 's' bit is set, it doesn't work at all. Or at least, it did not *used* to work before Os 3.9 latest BoingBag.

Why now does the freaking shell care about execute or not? That's again one of the sad stories of "bad design decision". Remember, commands have an input stream, and an output stream, you can redirect with "<" or ">". In  a sense, the shell has the same: It has an interactive input, which is the console the user types on and the shell connects the input of commands it executes to. This is the console. This stream is used for the "ASK" command, for example, to read user input. But the shell has *also* a command input, and this is where the commands for the shell come from. That's usually also the console, as the shell executes the commands as you enter them.

Now, however, if you execute a script (explicitly, or implicitly by a set s-bit), the shell does the following: It connects its "command input" to the file of the script. Sounds like the easiest thing to do: Magically, the input comes now from the script file. Since the interactive input is still connected to the console, the "ASK" command and all other commands still receive that as standard input, and hence read their input from the console as they should.

That's all pretty lean and neat unless... you want to execute a script from a script. Ooops, what do we do now, the CBM engineers said? They came up with the bloddiest ugly hack you can think of: Take the script to be executed, copy that to the RAM-disk (actually, T:), and append the rest of the script that is currently executed (and hence, the script that calls the other script) and append that at the bottom, then execute the new script.

That's just so wrong. I've written some hacky scripts, but even I wouldn't consider that a 'solution'.

Quote
See where I'm getting? Since the top of the current script is lost, and hence any "LAB"  command is lost, a "skip back" to a label that is below any execute command cannot see the "LAB" anymore, and hence, cannot skip back.

Luckely, the 3.9 shell is smarter, and does what every proper programming language does with recursion: It keeps a stack. Thus, yes, you can "skip back". In case you care, the stack is kept in some shell variables (you can see them with the "set" command, to be discussed in a later edition).

What the 3.9 shell does not is to "get away" with skip, and introduce *useful* flow control, as in "for" or "while". It would not be overly hard to do, but simply was not done. A bit of tradition still had to remain, I'd say.

Excellent summary Thomas, thankyou.

WaitX (Aminet) is also damned useful for creating cleaner scripts if anyone needs a bit more flexibility than 'wait' provides (often used with skip...lab).
« Last Edit: February 20, 2015, 02:22:38 PM by Boot_WB »
Mac Mini G4 (1.5GHz, 64MB VRam, 1GB Ram): MorphOS 3.6
Powerbook 5.8 (15", 1.67GHz, 128MB VRam, 1GB Ram): MorphOS 3.8.

Windows-free since 2011-2014 (Damn you Netflix!)
 

Offline TuKo

  • Jr. Member
  • **
  • Join Date: Sep 2013
  • Posts: 69
    • Show only replies by TuKo
Re: THORs Shell Hacks
« Reply #59 from previous page: April 20, 2016, 06:09:58 PM »
Dear Thomas,

I'm from the Apollo team and I'm following with attention your work on kickstarts modules.

As we are thinking to build a better ROM than we have today to give more functionnalities and bugfixes to users, I'd like to ask if you would kindly agree if we use your patches in our ROM ?

I mainly think :
* Shell 45.29
* layers.library 45.27

Thanks in advance