Reinvent the OS cont

[Iggy]

Linux is not well suited to hardware like the 68K.
And UNIX does do that well on low end hardware either.
I can't remember which, but there is a freeBSD or openBSD port for the Amiga.
But I would prefer a fresh approach.
I have used Microware's OS-9 on 68K based systems, and I have been pretty inimpressed with the open 8 bit versions of this OS.
We could start with a similar micro kernel and build out.
That, along with a lot of other advantages, would give us a much better priority based task switching system.
True real time performance could be realized.
The expansion bus is tricky.
In the 80's and 90's I used systems with ISA busses.
This is much easier to interface than PCI, but it is also a lot slower.
I have a few PCI bridge chips here that were designed to interface 68K and older PPC processors to a PCI buss, but its a complex device with a lot of software requirements.
You'd probably want to build a custom bridge chip using an FPGA.
Since its well known, an R200 video card would be a good first choice.
And sound from something simple like an SB Live audio card.

So, that would be the core I'd recommend.
'30 to '60 processor, FPGA PCI bridge chip, micro kernal based OS.
Euip that with some of the cards we are already using.
And there is your starting point.

[Iggy]

Hohum, you have much experience with Linux on m68k? It always worked well for me. There is both NetBSD and OpenBSD for Amiga too, at least old releases.

Yes, probably more than you, as I have been using 68000 processors since they were introduced.
And while linux may work for you, I'm not into pain.
Further, I've seen older implementations of Minix for the 68K that worked better than most Linux distros (and Minix isn't even meant to be anything other than an educational OS).

[Iggy]

http://www.osnews.com/story/28636/A_reimplementation_of_NetBSD_using_a_microkernel

That was interesting I thought.

However, for an Amiga experience I would never start with a Unix style kernel that insists on mapping each and every process in its own address space and using overlapping virtual address ranges. There are alternatives targeting 64-bit architectures, going under the label SASOS though many of those are not all that well grounded in the real world.

Personally, I think CPU evolution has not been a good match for the Amiga model: CPUs are going out of their way to separate and protect, and very few have mechanics for sharing. I have heard that the old PA-RISC did have the odd feature that was along that line.
The rise and power of FPGAs and what a single person can do on his own gives me hope that someone might force the transistors to yield to their will and make this something I'm looking for.

One of the features I liked about OS-9 68K was the use of reentrant code. Multiple processes could call executable modules with only one copy of the program in memory.
The 68K has limited memory addressing (in most versions). The more efficiently we use memory the better.
The techniques that will benefit our processor aren't common to other platforms.
That is why a generic OS like Linux or UNIX will never maximise performance on the 68K.

[Iggy]

Huh? First, in how far is this related to performance? It is probably related to available memory, but whether code is shared or not does not matter.

Second, Linux mmap's exectuables, which means that it only loads the parts that require execution, on demand, when needed. AmigaOs has to load the full thing to memory first. Worse, if you want to re-use executables between calls, they have to be written properly ("pure"), with no check of the Os whatsoever. Why that's an improvement I don't know.

Yes, the primary advantage IS the reduction in memory usage.
I started out using systems with limited memory resources, so I'm still focused on to tight efficient code.
"Pure", "No check of the OS at all.."?
No, the only constraint I know of is that they can't be self modifying (and any programmer that uses that trick deserves to be flogged).
And there is no constraint on OS interaction, I have no idea where you got that idea.
Ideally, there would always be a background process keeping count of they alloted timeslices that each process was allocated.
What I'm talking about does bring a few constraints to programming, but its also a lot tighter OS.

BTW - Since I have a Coldfire development platform on hand, I'm thinking about working on a kernel with that.

[Iggy]

No, not quite. Except you understand "self modifying" different from me. A program consists of three types of "memory regions", "code", "data" and "blank space" (bss). In principle, you can write to whatever region you like, but "code" usually contains the program binary, and modifying that by the program is what I would call "self modifying".

Unfortunately, that is only necessary but not sufficient for a program to be pure. It also must not write into the data or bss section, which - apparently - is not granted for your average program. Of course your average program wants to write to its data region, which typically contains C variables with program linkage (aka "global variables"), and of course, you want to write to the bss space (otherwise, it would just stay blank.).

So "pure" programs have to be a lot more careful. You essentially cannot use variables with program linkage (including library bases!), everything must be on the stack, or you must allocate from the heap yourself.

Some compilers come with startup code that create on each program start a copy of the data region on the heap, including manual relocation of the copy of the data space. Needless to say, while this makes programs pure (most of the time at least), it also causes additional overhead.   Why do you need a process for that? And keeping track of each single piece of memory a program allocated is usually too much overhead in first place. The way how Linux works is quite different. Every program gets its own heap, and can do with this memory region what it likes. The program usually includes compiler library routines that implement memory allocation from the heap. This memory allocation is not by the Os. The only exception is when the heap gets too small and requires resizing. Linux (and *ix operating systems in general) usually have a brk() or sbrk() system call to do that. But that's not a "memory allocator". It is usally several layers below malloc(), which is a C standard library function for maintaining the heap.  If the program ends, the entire heap is thrown away. If you start a new process, i.e. by the (infamous) fork(), the heap, stack and code are duplicated, and the separate copy continues execution behind fork(). (Actually, this is usually not a true copy, but rather depends on the "mad cow" trick, i.e. "copy on write").    Which makes it rather hard to implement any C type language on top of it, i.e. the "pureness" you seem to require. Bad idea, if you ask me.    

If I may, I would probably suggest a good book on operating system design.

You have an interesting fixation on your own definition of "pure". And my definition of self modifying code is completely valid, code that directly modifies  the module it runs in.  I never brought in data modules, and have no idea why you would.
As to OS structure, having used operating system before Linux even existed, I'm pretty comfortable with my knowledge level.
Further, nothing I have mentioned limits the use of variables.
One thing you seem to lack is a basic  understanding of kernel calls and operations.

So far, the most intellegent comment I have seen in this thread is the reference to DragonFly BSD.
The hybrid kernel used in that comes close to what I have suggested.
The micro kernel in MorphOS comes closer.

All I have suggested beyond that is reentrancy.
And that is not nearly the burden you seem to think it is.

Maybe you should examine how code used to be written. Then you might have a better understanding of this concept.

Maybe not, after all most Linux and UNIX users tend to gloss over the flaws inherent in their OS' of choice (like the disadvantages in using a monolithic kernel).

[Iggy]

And...to get past this point of contention, drop reentrancy as a goal.
The other points remain valid.
Like DragonFly, a nested kernel with a micro kernal core.
Basic rules that eliminate poor coding techniques that greatly complicate debugging (like self modifying code).

[Iggy]

No, not quite. Except you understand "self modifying" different from me. A program consists of three types of "memory regions", "code", "data" and "blank space" (bss). In principle, you can write to whatever region you like, but "code" usually contains the program binary, and modifying that by the program is what I would call "self modifying".

Unfortunately, that is only necessary but not sufficient for a program to be pure. It also must not write into the data or bss section, which - apparently - is not granted for your average program. Of course your average program wants to write to its data region, which typically contains C variables with program linkage (aka "global variables"), and of course, you want to write to the bss space (otherwise, it would just stay blank.).

So "pure" programs have to be a lot more careful. You essentially cannot use variables with program linkage (including library bases!), everything must be on the stack, or you must allocate from the heap yourself.

Some compilers come with startup code that create on each program start a copy of the data region on the heap, including manual relocation of the copy of the data space. Needless to say, while this makes programs pure (most of the time at least), it also causes additional overhead.   Why do you need a process for that? And keeping track of each single piece of memory a program allocated is usually too much overhead in first place. The way how Linux works is quite different. Every program gets its own heap, and can do with this memory region what it likes. The program usually includes compiler library routines that implement memory allocation from the heap. This memory allocation is not by the Os. The only exception is when the heap gets too small and requires resizing. Linux (and *ix operating systems in general) usually have a brk() or sbrk() system call to do that. But that's not a "memory allocator". It is usally several layers below malloc(), which is a C standard library function for maintaining the heap.  If the program ends, the entire heap is thrown away. If you start a new process, i.e. by the (infamous) fork(), the heap, stack and code are duplicated, and the separate copy continues execution behind fork(). (Actually, this is usually not a true copy, but rather depends on the "mad cow" trick, i.e. "copy on write").    Which makes it rather hard to implement any C type language on top of it, i.e. the "pureness" you seem to require. Bad idea, if you ask me.    

If I may, I would probably suggest a good book on operating system design.

You have an interesting fixation on your own definition of "pure". And my definition of self modifying code is completely valid, code that directly modifies  the module it runs in.  I never brought in data modules, and have no idea why you would.
As to OS structure, having used operating system before Linux even existed, I'm pretty comfortable with my knowledge level.
Further, nothing I have mentioned limits the use of variables.
One thing you seem to lack is a basic  understanding of kernel calls and operations.

So far, the most intellegent comment I have seen in this thread is the reference to DragonFly BSD.
The hybrid kernel used in that comes close to what I have suggested.
The micro kernel in MorphOS comes closer.

All I have suggested beyond that is reentrancy.
And that is not nearly the burden you seem to think it is.

Maybe you should examine how code used to be written. Then you might have a better understanding of this concept.

Maybe not, after all most Linux and UNIX users tend to gloss over the flaws inherent in their OS' of choice (like the disadvantages in using a monolithic kernel).

[Iggy]

Oh lordy, please go on fantasizing about the DragonFly "micro kernal", it sounds orgasmic!

Actually, micro kernel OS' are no fantasy. QNX, DragonFly, Microware OS9, MorphOS, and almost all process control OS' (as well as plenty of other examples).
And unlike OS' based on a monolithic kernel, many of these can operate in true real time.
Another feature, which Linux has just caught up to, is the ability to dynamically load and unload drivers (much easier when the drivers reside outside the core of the kernel).
And btw, I have plenty of experience with Linux, which is why I am not fond of it.
No great surprise that an Amiga user would be satisfied with Linux as the Amiga never had real time response or stable operation either.
And if we are to continue a legacy of tight, compact software then Linux is definately to be avoided.
I grew up during the period Bell Labs and our nations universities were developing UNIX. Linux is little more than the illegitimate child of UNIX.

Don't you think we can do better than a OS with a 50 year history?
Linux reminds me of Java or Flash in that it isn't that great, but its momentum seems to keep it going.

And you guys can try to attack my credentials, but I was building SWTPC equipment before most of you were born.
Before we really had OS'.
I have worked for a company that built its own 68k based system.
And I had a hand in writing the video drivers for that system and porting a GUI to it.
So...diddle around with your Amigas as much as you like (I'm fond of the platform too), but I have already seen ways where 68k based systems could be improved.

[Iggy]

Now...could we get ON with this discussion.
I am sure the majority of us would be unwilling to trade what we have for yet another Linux derivitive.
So, if we assume a fresh start, with some coding done at levels as low as assembly, all in the name of efficiency - what features would that OS have?

I have already outlined my ideas.
How about a few from all of you that don't suggest a retread of something tired?

[Iggy]

BTW - You all ARE aware that Amiga OS is considered a micro kernel OS, aren't you?

[Iggy]

There was a lot of advanced topics you covered. Good job. Could you just pick an OS that has the most software already available?

Well heck, then we'd have to go with Windows, OSX, or Linux, and I don't want to do that.
Even Amiga OS has a better software base than what I'm suggesting.
This is where Thorham and I agree, the lack of software would be a major disadvantage.

[Iggy]

This is not my definition of "pure". It is the AmigaDos definition of "pure". Sounds strange? Maybe. But I did not design this stuff.

My apologies. I am not here for contention.
Obviously much of what I am considering would definitely be "impure".
And after thinking about your posts, reentrancy isn't really a necessity with as much memory as we can use.
With a 1.5 GB memory range, and an efficient processor, the added complications may not be worth it.

[Iggy]

Iggy, I have issues with you calling the DragonFly kernel a micro kernel, it is not a micro kernel design, it is a hybrid at best, with some features most known from micro kernel environments. If DragonFly is a micro kernel, then FreeBSD is too, or any *ix that pushes features out in userland. Do you consider Darwin a micro kernel OS? It used to be.

I find it funny how DragonFly keeps coming up as a buzz word among amiga users, but how many of those mentioning it has actually used it?! For the record, I have, at work even, evaluating HAMMER for possible use in production environments. Eagerly awaiting HAMMER2 to evaluate for a possible multimaster cluster FS solution.

Yes, its not a true micro kernel. It has layers. I'm used to that with other true micro kernel OS'. I've mentioned OS9 before, its kernel grew into two distinct segments in its last 8 bit version and remains fairly large in the 68k version.
But your right, DragonFly really stretches the definition, and its more of a hybrid.
And its not the first UNIX or Linux variant that has tried this.
Still, if we were to approach this it would have to be a clean start.
Really all we are talking about is applying so of the  improvements that have become common in process control OS' since Amiga OS was created.
We can definately improve on real time response and reliable multi-tasking.
Having a GUI that was OpenGL ES compatible would be nice.
I don't know enough about audio standards to make any real suggestion, except maybe to have a coprocessor to help lighten that task.
Other issues, what processor ('030, '040, '060, or FPGA), what type of memory, and what kind of expansion bus?
So, again, any ideas?

All I'm sure about is the kernel, since a micro kernel is our legacy, our current reality (as in MorphOS, and probably OS4), and our likely future.

Oh, and one suggestion for video.
While its old, we know enough about the R200 to make that an easy target.

[Iggy]

Understandable Staf,
I'm still unsure that a break with compatibility is such a hot idea.
I'm a bit of a Neanderthal when it comes to MorphOS.
I'm in no hurry to see an ISA change, and I'd like to see Abox remain intact while we pursue a second concurrent environment that could make use of the features that would break compatibility (SMP, larger memory maps, etc).

Hey Kolla,
What is an exokernel? I don't think I have heard that term before. Educate me.

Oh...I looked that up. Its a very apt description for the Amiga kernel.

[Iggy]

I like the OS 9 microkernel approach. Should you have software to run on it as its ready? E.g. goal:
Networking
File manager
are some suggestions.

We had those running in the early 90's.
What we could have used was a better GUI.
We ported an X-windows based package called G-Windows that had been created by Steve Adams for a company that specialized in process control systems, Gespac.
It worked, but like a lot of Linux (and virtually all UNIX) derivatives you still had to use the CLI to get some things done.
Personally, while I can see the utility in a terminal window, I'd prefer that it be the instrument of last recourse (you should be able to do virtually everything you need through the GUI).

Oh, and with all deference to Staf, I'm still farting around with obscure micro kernel OS'.
We have to remember that Linux only makes up about 1% of the marketplace.
BSD, considerably less.
Amiga like OS'? Probably an insignificant percentage.
So what?
Does that mean I have to use a more common solution?
No, not if what I am using provides me with the utility I need.

I'm sorry to see good, committed AROS developers dropping out just as the project is reaching completion.

Drawing users to a new 68K OS? Probably not too realistic, but I'd like to have it to play around with. And a Coldfire version would be an easy port.
With the Coldfire V1 cpu available as a royalty free FPGA core and cores for the 68K steadily improving, future hardware could be built with better performance than our legacy hardware.