How can executables work when being thrown into different addresses on 68000 ?

[itix]

It doesn't require fixes to Intuition, it merely requires that you allocate the memory as public. It would be entirely possible to enforce it for new programs and not for old programs, by forcing old programs to have MEMF_PUBLIC for all their allocations.

Unfortunately your operating system stops functioning if structures are made private. So you have to fix Intuition and it would be better if the software was not using this low level component in the first place. Many of its design issues could be hidden to a high level abstraction layer.

But it is academical anyway because we know nobody is ever going to create an OS with MP compatible with existing software.

[Fats]

In MP system there should be no concept of public memory at all. Public as in "memory that any task can alter at any time".

I don't see what is wrong with providing public memory in a MP OS. Programs should be able to not need it or use it though.

There is no true memory protection if task Y by accidentally writing data to address X can cause entire operating system to crash. For example try memset(SysBase, 0, 0xffffffff) and see if the system can survive from that.

SysBase could be read-only for the MP programs and they will just be killed if they try to do the above. The non-MP programs can still step on each other toes but won't be able to bring down the MP ones and the part of the OS that is also MP converted. I don't see why some clever MMU trickery shouldn't be able to do that.

I would just forbid use of low level APIs like Intuition and migrate software base to use higher level abstraction (i.e. MUI).

I would not go that far and you also want to retain the possibility for devs to program their own gadgets which will need lower level functions. I don't want the GUI framework become part of the kernel.

One other thing I also certainly want to keep is messaging by pointer passing. IMHO an OS not doing this can't be called an Amiga-like OS.

Introducing more memory types is just insane adding more unnecessary complexity. AllocMem() with MEMF_ANY should be enough.

I personally want some more fine-grained access rights than the classic process based MP as in UNIX with an additional kernel context that basically can read/write everything. I also would like something that is much less complex than SELinux for defining the access rights.
To me an amiga-like OS has to be micro-kernel and the kernel (e.g. exec.library) should be kept as small as possible. It should just provide the facilities to two tasks to have memory which they both can access but no other tasks can access but the kernel's involvement should stop there.

[donpalmera]

The reason for asking is that a lot of devices use ARM and MIPS processors that lack any MMU.

** snip **
 This usally means only µCLinux remains as a free OS choice.

These days in situations where you don't have an MMU you usually don't have one because you aren't going to be running a general purpose OS. You are either going barebones there or using an RTOS. Most RTOS don't have loaders for executables etc because the "programs" are compiled into the same image as the "OS".

load arbitrary code into arbitrary locations.

Surely that requirement exists for any OS that can load programs at runtime.
The code isn't arbitary though. If it's going to be running with an OS it has to abide by the requirements of that OS.

Decoding ELF binaries with relocation table and making use of it seems however way more complicated than the Amiga Huff format.

I'm not sure how far you've actually researched this stuff but uClinux doesn't use ELF either. It uses BFLT. Because of how hard it is to get a target into gcc etc and have it stay working the uClinux targets do compile to ELF but that output is then passed through a tool that generates the relocation table, adds information about the entry point, stack size, whether the code should be loaded to into ram or execute in place etc. The BFLT is a lot simpler than HUNK.. It's basically a header, relocation data and then the program image.

[donpalmera]

I also would like something that is much less complex than SELinux for defining the access rights.

SELinux has little to do with memory protection in the context of this thread.
Unix has simpler access control systems.. you know like the one that's built in i.e. file permissions etc.

[donpalmera]

AmigaOS changes them for every loaded program.
 You missed his point.

I don't think I did. You seem to have thought that I was talking only about AmigaOS when I wasn't. Not sure why I would bring up microcontrollers if that was the case TBH.

Most operating systems that use an MMU have separate memory spaces because it has other benefits, like avoiding memory fragmentation.

And being able to resize tasks etc. Memory protection goes hand in hand with what a MMU is doing in most systems but you don't have to have a fully fledged MMU to protect the kernel from user processes.

If you don't have an MMU then you can't use separate address spaces or realistically protect memory between tasks.

If you don't have an MMU but have some sort of memory protection you can:
Protect the kernel and it's memory. Protect memory mapped devices like NOR flash from being corrupted by a runaway user process. Depending how how fine grained it is you can protect other processes from the active process by restricting the active process to working within the memory allotted to it. I know a lot of that doesn't apply to AmigaOS.
 

Your limited memory protection sounds like the TLB in the 68EC040, which isn't really good enough for what AmigaOS uses the MMU for.

My limited memory protection is the type of thing that exists in a lot of recent microcontrollers and to a very limited degree in the DragonBall SoCs if it has to be 68k related.

[itix]

I don't see what is wrong with providing public memory in a MP OS. Programs should be able to not need it or use it though.

The operating system should not have system critical structures in public memory. It should be possible make memory shared with other processes to at request but not by default.

SysBase could be read-only for the MP programs and they will just be killed if they try to do the above. The non-MP programs can still step on each other toes but won't be able to bring down the MP ones and the part of the OS that is also MP converted. I don't see why some clever MMU trickery shouldn't be able to do that.

Easier said than done. Libraries manipulate libbase data so there must be way to unprotect libbase temporarily. Doable but again you have to go through system code to fix it. We could accept that SetPatch() would not be permitted anymore (since it assumes public memory model) but the OS is using SetPatch() at few places so you need to redefine some other code again (mostly locale/utility stuff, IIRC).

You can always find a solution to one problem but it is likely to introduce two more. The sun explodes before all that code is fixed and in place with backward compatibility. IMO

I would not go that far and you also want to retain the possibility for devs to program their own gadgets which will need lower level functions.

You dont need Intuition for that. Its way to define gadgets is not MP friendly anyway. It is tricky when user application structures become part of system structures like struct Gadget would be. This could be fixed by copying data to system private structures but then you will find yourself rewriting bulk of Intuition again...

Or alternatively run Intuition in user space so all structures would be private. But then we could use something more sophisticated UI framework.

I don't want the GUI framework become part of the kernel.

It doesnt have to. Many BOOPSI based GUI framework could easily run in user space. There is enough abstraction to hide system internals what Intuition does not.

One other thing I also certainly want to keep is messaging by pointer passing. IMHO an OS not doing this can't be called an Amiga-like OS.

That is no problem but messages should not be public to everyone. To support this scheme messages should be allocated in a manner that they can be shared with target task (as designated by mp_SigTask) but not all message ports have designated target task.

On AROS mp_SigTask is even designated as union with callback pointer (when you PutMsg() to this kind of port OS calls this function). This PA_CALL extension is very neat feature but not actually MP friendly. I think it is perfect example when shared memory model encourages defining features which are difficult to replace without breaking compatibility both binary and source level.

[freqmax]

@donpalmera, I want to be able to run a partial general purpose OS. Such that one can login edit C-files and compile them. Mostly to be able to change functionality without setting up a cross development environment. Which is not always possible.

Code loading at arbitrary positions are done by miost OS:s. The difference is that computers with MMU can use a static memory position as opposed when you actually need the relocation table.

I found the x86-32-ELF format more complicated than Amiga-68k-HUFF. I didn't have a look at µCLinux in that aspect.


As for making use of MMU-less ARM (or perhaps MIPS). It's just noted that ELF decoding and relocation might be less straightforward than I thought. But most importantly is that I know gcc + ld may and will produce relocatable binaries. How to make it happen in code is another story. The next chapter is how to make multitasking happen. And perhaps if a sufficient handleable QFP surface mount CPU with MMU can be had.

[bloodline]

@donpalmera, I want to be able to run a partial general purpose OS. Such that one can login edit C-files and compile them. Mostly to be able to change functionality without setting up a cross development environment. Which is not always possible.

Code loading at arbitrary positions are done by miost OS:s. The difference is that computers with MMU can use a static memory position as opposed when you actually need the relocation table.

I found the x86-32-ELF format more complicated than Amiga-68k-HUFF. I didn't have a look at µCLinux in that aspect.


As for making use of MMU-less ARM (or perhaps MIPS). It's just noted that ELF decoding and relocation might be less straightforward than I thought. But most importantly is that I know gcc + ld may and will produce relocatable binaries. How to make it happen in code is another story. The next chapter is how to make multitasking happen. And perhaps if a sufficient handleable QFP surface mount CPU with MMU can be had.

Compile your code with the -fpic option ;-)

[psxphill]

Unfortunately your operating system stops functioning if structures are made private.

I'm not 100% sure what you mean.
 
If by that you're saying that not specifying MEMF_PUBLIC on memory that needs to be accessible by another task will cause the system to become unstable, then I agree. However we're not discussing that issue at all.
 
The operating system stops functioning if a program calls Disable() or Forbid(). Memory protection would prevent some accidental or malicious memory accesses from going unnoticed.
 
On 68000 you can do similar bad things to the operating system by using odd addresses.
 
Arguing for a robust operating system where one task can't take down the whole machine is a whole different ball game & trying to solve those issues would involve throwing everything away and starting again.
 

If you don't have an MMU but have some sort of memory protection you can:
Protect the kernel and it's memory. Protect memory mapped devices like NOR flash from being corrupted by a runaway user process. Depending how how fine grained it is you can protect other processes from the active process by restricting the active process to working within the memory allotted to it. I know a lot of that doesn't apply to AmigaOS.

If it's fine grained enough that you can protect the active process then it's basically an MMU. Protecting a handful of ranges of memory doesn't count as memory protection (AmigaOS already does that itself).
 

The difference is that computers with MMU can use a static memory position as opposed when you actually need the relocation table.

I've yet to see an operating system that uses an MMU to avoid ever having to re-locate executable files. The way windows executables works is they have an address they are linked to, but there is enough information that they can be relocated. If it can load it to the address the file wants then it saves time relocating it.
 

That is no problem but messages should not be public to everyone. To support this scheme messages should be allocated in a manner that they can be shared with target task (as designated by mp_SigTask) but not all message ports have designated target task.

You can never can know what task needs to be able to write to your memory. If you use Read() to fill your message then the memory needs to be writable by a task created by the filesystem. To do what you propose in every circumstance isn't simple, although it could be used in some cases.

[freqmax]

Suggestion, make new OS. And write an compability layer.

[Fats]

You can always find a solution to one problem but it is likely to introduce two more. The sun explodes before all that code is fixed and in place with backward compatibility. IMO

I just hope I somewhere find the time, inspiration and transpiration to prove you wrong

[Fats]

Suggestion, make new OS. And write an compability layer.

Linux+ AROS ?

[itix]

I'm not 100% sure what you mean.
 
If by that you're saying that not specifying MEMF_PUBLIC on memory that needs to be accessible by another task will cause the system to become unstable, then I agree. However we're not discussing that issue at all.

No, you did not understand it at all. The operating system need that tasks allocate some memory with MEMF_PUBLIC to be accessible by operating system tasks and handlers. There is no clean separation of user space and kernel space.

The operating system stops functioning if a program calls Disable() or Forbid(). Memory protection would prevent some accidental or malicious memory accesses from going unnoticed.

Just try something like this:

int main(void) { AddPort(malloc(sizeof(struct MinNode))); return 0; }

Next call to FindPort() may or may not hang.

You can add memory protection as much as you want but with this code operating system stops functioning.

Or maybe just:

void *gadget = AllocMem(sizeof(struct Gadget), MEMF_ANY); // Memory type is not documented in AddGadget() !
InitGadget(gadget); // init gadget to sane values so it wont crash imemdiately
AddGadget(winptr, gadget, 0);
FreeMem(gadget, sizeof(struct Gadget));
Wait(0);

You will find out that input.device is reading from unallocated memory and your keyboard and mouse is dead. Our software, however, is still runnable. It is also possible input.device is now trashing some innocent memory.

At any time we can inject toxic structures to the system. We dont have to do it deliberately: it could be just an error in the program logic.

Arguing for a robust operating system where one task can't take down the whole machine is a whole different ball game & trying to solve those issues would involve throwing everything away and starting again.
 

Isnt this what memory protection should do? Anything less is just fake if we dont count drivers and such.

[Fats]

SELinux has little to do with memory protection in the context of this thread.

I have to admit I am no SELinux expert (I just disable it) but what I had in mind when I wrote that is that you use shared memory if two processes want common memory access and SELinux is used to restrict access right to this shared memory.

[guest11527]

Probably a couple of concepts are mixed here. Please allow me to explain:

1) Memory *protection*: If you mean by that that certain areas of memory can be prevented from getting overwritten, yes, this is perfectly possible. If you load kickstart modules with "LoadModules", you can protect them from getting overwritten by MuProtectModules, this is what the command is good for. If you write a program and want its code segment protected from getting overwritten, you can do so by post-processing with MuLink. All the features are provided by the mmu.library back then.... of course, nobody ever used MuLink, but MuProtectModules is in active use.

2) If you mean *isolation* of programs, i.e. protect the address space of program a) from messing with the address space of program b), then there are only limited means (but *some* means) to do that, but programs need to request this explicitly. Problem: The Os moves its data structures around in global memory and has no resource management. Meaning, it has no control which process "owns" a resource. Thus, every program that interacts with the Os *necessarily* has to see parts of the global address space simply because that's where the Os keeps all its data. There is neither a problem with program a) allocating a resource, passing it over to program b) which uses it, and program c) releasing it. As long as this is consistent.

3) Can programs still isolate *parts* of their data? Yes, they can. Unfortuntely, this part of the data cannot be used to store any type of Os structures, only private data. But still. The corresponding concept is a "MMUContext", and again, it is provided by the mmu.library. A program thus can setup a context, shares parts of the global (default) context with the Os structure with other programs, but protect its private data from other programs running in other contexts. Again, the concept is there, the software that provides such magic is there (including context switches, context creation and administration) just that nobody uses it for that. Private memory is that - private. It should not have MEMF_PUBLIC set, it does not exist in exec, but the memory.library can provide such memory. The memory.library *also* exists, just no programs using it.  

4) Is virtual memory possible? Not for Os structures. Only for private memory. Reasons are basically that swapping in memory requires a task switch, thus breaks any Forbid() and Disable(), and the Os is not prepared for that. However, if a program uses private memory (via the memory.library) such memory *can* be swapped out, and loaded from disk (automatically), again - of course - breaking the Forbid() state. Just that there are no programs using the mechanism.  Thus, a lot of plans and a lot of mechanisms have been created, but since the software basis and the user basis broke away, all the mechanisms and constructions remained unused.

[Fats]

3) Can programs still isolate *parts* of their data? Yes, they can. Unfortuntely, this part of the data cannot be used to store any type of Os structures, only private data. But still.
...

4) Is virtual memory possible? Not for Os structures. Only for private memory. Reasons are basically that swapping in memory requires a task switch, thus breaks any Forbid() and Disable(), and the Os is not prepared for that.

I also think most of these restrictions can be removed by doing the proper changes in the OS and most of the OS can be MP also.