What is memory protection and why is it so hard to implement for the AmigaOS?

[itix]

It is not just that it breaks existing software. It is not possible implement memory protection without complete redesign of API.

[itix]

I disagree here. I think it would be possible to have minor changes to the API and provide memory protection to programs that follow certain policies. By the latter I mean no data allocation on the stack that has to publicly available, obey the MEMF_PUBLIC/MEMF_PRIVATE meaning, etc.

Why this MEMF_PUBLIC insanity? There should be no reason why application should use this stupid brainfart. Remove memory usage restrictions and you are closer to real MP.

Of course all software I have written for Amiga allocates data on the stack and never cares about MEMF_PUBLIC. Believe it or not this coding practise works just fine. It works just fine even on Linux and Windows.

Programs could in the executable or during start-up indicate that they follow these policies (with the consequence they may be killed when not following them). Programs not having this indicator won't be able to have MP.
For this to work you have to implement a MP that fits with the Amiga API. This means not trying to enforce the Windows/Linux MP with address spaces for each process/task but getting your inspiration from single address space operating systems and similar systems.

It could be done using UAE kind of solution. Run every program in its own sandbox.

The main extension of the API would consist in extending the memory pools API so programs can indicate which other programs may access certain memory pools in read and/or write mode.

This is only minor detail. It does not fix the OS.

[itix]

@Fats

Then you are not implementing an amiga OS anymore IMHO. One thing - again IMHO - that at least is needed for an amiga-like OS is message passing by pointer passing and for that to succeed you need to have single address for the memory accessible in between different protection entities.

Message passing is only minor detail in OS design. What about features like AddPort(), AddSemaphore(), AddTask() and other system calls? How do you protect system against rogue applications? I could add dummy uninitialized message port with invalid name tag. The entire OS could crash when it scans ports next time.

You can of course deprecate those functions. But then we are not talking about minor changes anymore. We are also having problem with layer hooks, global semaphores and other design flaws (from MP POV) added in Kickstart 2.0. Shared address space design is present everywhere.

Also to be able to have stack extension no data that is to be shared may be allocated from the stack. This way the stack would be private memory space and it could have the same vritual address or overlapping addresses for different programs.

And how are you going to implement stack extension with limited 32bit shared address space?

No, because you can't send messages in between the sandboxes.

You would obviously need an API extension to communicate safely with boxes. New applications would have to support it. Old applications could share sandbox.

[itix]

Many of these can run as separate tasks/threads in the same "sandbox" as the applications that uses them. It is not necessary for the console.device to run in it's own address space on Amiga OS for example. Many other pieces of system code can reasonably run in the applications address space, further limiting the types of messages that needs to be possible to bridge to other processes.

But you would have to start new instance of console.device for each sandbox. And not just console.device but almost every other OS service. Drivers must get moved out of sandbox of course.

What about USB stack? I dont think it could stay inside this sandbox. But if it is outside sandbox how it runs? How do you modify USB stack to run there? Or is it just its own box? How about its own drivers?

It wont be easy

So yes, you could add a dummy uninitialized message port with invalid name tag, and in the proposed model it would take down the tasks running in that one process for that one application, instead of taking down the entire OS. For "cross-process" message ports the kernel would necessarily need to do careful validation of all arguments.

I dont think old PutMsg() would be suitable for cross process communication anymore so you would define new API to do that, anyway...

[itix]

Aaaeee, buggy forum software deleted my posting.

So only short question now: if USB stack mounts a USB masssstorage device woudnt filesystem for that USB drive run in UBS sandbox context?

[itix]

In a MP program the stack can contain only data private to the program that can never be exported to other programs. This way all stacks of all programs could start on the same virtual address.

This causes serious side effects. Consider following example:

void foobar(char *name)
{
   struct FileInfoBlock fib;
   TEXT buffer[108];
   BPTR fh;

   strcpy(buffer, name);
   strcat(buffer, ".txt");

   Printf("Open file %s\n", buffer);

   fh = Open(name, MODE_NEWFILE);
   Write(fh, buffer, sizeof(buffer));
   ExamineFH(fh, &fib);
}

Looks like you have to fix dos.library first. You are going to have other indirect implications. Think about this:

void PlaySound(BYTE *buffer, LONG bufferlength)
{
   io->Command = CMD_WRITE;
   io->Buffer = buffer;
   io->Length = bufferlength;
   DoIO(io);
}

Ooops? You must document what API calls can take data stored on the stack. This is why I dont like MEMF_PUBLIC crazyness.

[itix]

I did not say that you could use libs without any change. And it will not only be dos but also exec, gfx, intuition, ...
In this example I don't see anything that can't be handled by dos internally, although the proof will be in the pudding.

It must be handled by dos internally. It is way too much work for user code to always copy filenames etc to temp buffer.

It is about how to put public data on stack and make the stack auto-extending. Do you have another solution for that ?

Go 64 bits ;-) There you should have got enough address space for next 10 years...

But anyway... experiment it and tell us how it went ;-)