Coldfire AGAIN

[HenryCase]

Einstein wrote:
the cursed Winblows (and Linsux) does not permit the sweet buggy apps destroy the data of the others (and consequently mine) and kill the OSs, what a shame.

When was the last time this happened to you, and what version of AmigaOS were you using at the time?

[HenryCase]

@JetRacer

The different versions of AmigaOS aren't equally stable, so my question still stands.

[HenryCase]

Einstein wrote:
And there is no difference between any AmigaOS since *there is no memory protection*.

Not true. AmigaOS4, MorphOS, AROS64 all have at least partial memory protection. I am currently investigating how easy it will be to add full memory protection to AROS (without throwing away AmigaOS 3.1 API compatibility), and so I asked you which version you had used so I could get an idea of the stability compared to the other memory protection options.

So I'll ask again, which version of AmigaOS were you using?

[HenryCase]

bloodline wrote:
The answer is: Not very. i.e. Impossible. While the exec design team clearly envisiaged memory protection at some future time... none of the rest of the OS design team did (I think intuition alone is a very bad boy!), and public/private memory flags were never enforced.

Tasks and libraries happily pass memory pointers around without a care in the world.

That is useful information to me bloodline, so thank you for that. None of what you describe makes retrofitting memory protection impossible though. Let me explain...

The memory doesn't have to flag whether it is public or private as long as the API functions asking for memory space are not allowed direct access to the memory. You put an OS layer inbetween memory allocation functions and the real memory. This MMU gives the memory allocation functions the impression that it is writing directly to the memory, when in reality it is controlling memory allocation/deallocation and protecting unwanted memory access.

For those memory functions called without a flag, the memory status is set to private by default. You then have the issue of inter-program communication. For this, you want to allow the programs to attempt communication, but you want to prevent incorrect values being entered into the program space and potentially crashing the OS. You allow the value to be passed by the first program, but the MMU checks to see if the value is valid before it passing it on to the second program.

For the few programs where this method is not suitable (where you need as much speed as possible for example) the MMU can either be switched off or the program can be patched. The MMU will only be known to the new kernel and the user, the API functions will not be aware of its existence, so they will not mind using non memory protected mode.

Those are my ideas as they stand at the moment. As I said I am still conducting research into the feasibility of a memory protected AROS.

Do you think my ideas are sound in principal?

[HenryCase]

bloodline wrote:
MMU's are hard to get your head around at first but some solid study and a lot of running it out on paper with a pencil will give you a good idea of the issues here...

I am studying memory protection, but as you failed to grasp my ideas correctly allow me to try and explain them again...

You can't just switch off the MMU once it is on... the whole integrity of the memory structure is maintained by this device... Switching it off would leave the memory in effectly a random jumble of 4k blocks that used to mean something...

Yes you can. Imagine this scenario. You have two kernels available MP-Exec (Memory Protected Exec) and Exec (the standard no memory protection version). You've started running a program, but need it to run faster. When you give the command to switch from MP-Exec to Exec (I know this would remove memory protection for all processes), what effectively happens is that the MMU updates the 'virtual' memory addresses with the real memory addresses (like saying to the program 'here's where your data was'). Once this process is complete a kernel switcher program takes over, freezing all processes and keeping the memory as it was (kind of like when you put a laptop into hibernate). The kernels are switched and the processes and memory are unfrozen. The programs don't see any difference because the APIs of MP-Exec and Exec give the same results.

I really can't see a problem with what I've described, perhaps you can?

One of the great things about AmigaOS is that is places no restrictons on what is passed in an exec message... In effect the OS has no idea what the programs are sending to each other. (it's the same problem as the parasite emulator we discussed before).

But this can be changed by adding an MMU layer to the kernel and updating the kernel functions, so it's not an issue. Of course no memory protection gives you more freedom, what I am describing is a way to add memory protection for those who want it (which seems to be a lot of people).

bloodline wrote:
Now here is where the problems begin... You have two tasks, both have their own address spaces... one task sends a pointer to a data block to another task in a message... that pointer is meaningless to the receiving task, since the pointer is either not vaild in its address space or pointer for somthing completely different!

As the MMU is dishing out memory, it knows what kind of data will be valid for the space. So, for example, if you try to pass a message suitable for a  graphics routine into a memory space designated for sound functions then the MMU will not allow it. This is just one example, there will be other protection measures in place, such as if a program crashes you could limit the crash to its own allocated memory space, seeing as all reading and writing of memory goes through the MMU.

bloodline wrote:
The only way around this is to have a public pool of memory mapped to the same address window in all task address spaces... but since the public/private flags were ignored this can't work.

FYI, that's not what I'm trying to propose.

bloodline wrote:
I don't understand... The way memory protection would work is that each task gets it's own address space... in effect each task gets up to 4gig of address space to itself. I shall expalin the reprocussions of this later...

Don't understand? Okay, this is how it works...

When an API call requests a memory space (doesn't matter if it has flagged anything) the MMU layer of the OS looks for a suitable space to store that memory and then tells the API call (I stored your data 0x001 here: 1F2C, or whatever). The point is, that doesn't have to be where the data is stored, the app just has to think that's where it is stored. The MMU has a look up table linking the 'virtual' location with the real memory location. When the memory stored at 1F2C gets called the MMU then fetches the real data...

My laptop battery is about to die. I do have more to say but I'm hoping the penny has dropped by now. I'll be back later, feel free to comment on what I've said so far.

[HenryCase]

Karlos wrote:
How do you take, say, 16 4KiB pages scattered across the 4G physical address space that an application requested and originally thought was one contiguous 64KiB lump of memory and tell it "here is where your data was" ?

By arranging it into a 64KiB lump before you give memory control back to the program.

Karlos wrote:
A single allocation of memory on a VM system using an MMU that an application uses a single pointer to refer to can translate into many unrelated chunks of genuine physical memory. You can't assume contiguous address mapped memory is contiguous in physical RAM.

Give me an example of when you'd use a pointer to address more than one memory location so I can explain how its done.

[HenryCase]

Kronos wrote:
@HenryCase

Problem is the OS can't decide wether one task made a legitemate request for data owned by another (lets say stuff running inside the input.device-task) or if it is just some roque pointer .....

I am not asking it to. I am asking the OS to make some limits on the commands that are physically not possible, like trying to put a 6-bit instruction into a 3-bit space. It was just one example of how the memory layer would keep the system stable.

Kronos wrote:
Next idea is usually to just copy system-structures into each tasks private part of memory, but once you take a look into the interlinked (and open) system-structures you'll notice that ain't pratical either.

Please expand on that point so I can understand it.

[HenryCase]

HenryCase wrote:
My laptop battery is about to die. I do have more to say but I'm hoping the penny has dropped by now. I'll be back later, feel free to comment on what I've said so far.

Continuing from where I left off...

Thought of an analogy about API calls that may be useful. In a restaurant, think of the API calls as the customers and the memory management and kernel as the waiter and chef. The customer can make any number of rediculous requests about what they'd like to eat, but in a restaurant it's the chef and waiter that decides what they're having. A non memory protected OS is like a restuarant without a waiter, the customers have to go into the kitchen to get their food, and can make a mess. If you put the waiter as a barrier between the kitchen and the customers then they can't make a mess in the kitchen. Hmm, must have food on the brain!

The point is API calls are not in control of the system, they are only means to make requests to that system. As long as the waiter delivers the same food that the chef was providing before the customer is not going to complain. Capiche?

Let me set a challenge. I've hopefully explained a little about how this memory abstraction layer will work. I'm going to stop calling it a MMU now and call it a MAU (Memory Allocation Unit) because although I am confident my ideas will work I don't know if this is how MMU's traditionally work.

My challenge is this. I have a program running which has got an important string "ABCDEF" stored in memory at real location "0x001" and virtual location "0x0FF". Using your knowledge of the Amiga's API architecture, I want you to try and corrupt this data. I am confident that whatever you suggest can be dealt with without breaking AmigaOS compatibility, so try and prove me wrong.

[HenryCase]

AJCopland wrote:
Wow you can do that? (note: this is sarcasm)

Think of it like HD defragmentation if it helps you understand it.

AJCopland wrote:
With all due respect Henry both bloodline and karlos are trying to explain how MMUs work from a position of knowing, at least from a code point-of-view, how they function. So some of your replies have read as being rather rude.

I'm not trying to be rude, but it's hard to maintain composure when you are being misunderstood. As I've just mentioned, lets call my idea an MAU (Memory Allocation Unit) as apposed to a MMU as I don't know if what I'm describing is your daddy's memory management system, if you catch my drift.

[HenryCase]

@thread

First of all, I'd like to thank those that have responded to my memory protection posts, I've got a lot to digest there, and will do ASAP. However, it is late so I'll concentrate on bloodline's post quoted below, as it seems the most concise.

I would also like to state that I think memory protection would be useful with Coldfire, especially if we want to move away from emulating every 68k instruction.

@bloodline

bloodline wrote:

HenryCase wrote:
My challenge is this. I have a program running which has got an important string "ABCDEF" stored in memory at real location "0x001" and virtual location "0x0FF". Using your knowledge of the Amiga's API architecture, I want you to try and corrupt this data. I am confident that whatever you suggest can be dealt with without breaking AmigaOS compatibility, so try and prove me wrong.

Task A Stores "ABCDEF" at 0x0FF.
Task A sends a message to Task B.
The message simply contains 0x0FF.
Task B looks at Address 0x0FF and it doesn't find "ABCDEF".

Reason:
0x0FF are two different Physical memory locations to the separate tasks. infact Task B might even have some code at its 0x0FF, or anthing!!!

The OS has no idea that Physical address 0x0001 has to be mapped to the same place in both tasks... in fact it might not even be possible to map that physical address to the same virtual address!

Neither of the Tasks can ever know the Physical address, as their 0x0001 is mapped somewhere else... different for each task!!!

So if I understand correctly you're saying that the data cannot be read by more than one task because the OS doesn't know how to share this information?

If I have understood correctly, allow me to use your example  but fix it with the MAU (yes I am still going to use that term to avoid confusion):

Task A requests to store "ABCDEF", the MAU stores it at location 0x0001 and passes back a pointer to that location through assigning a 'virtual memory space' 0x0FF.
Task A sends a message through the MAU to Task B to look at memory space 0x0FF.
Task B makes the request to look at address 0x0FF, the MAU does not see a reason to not show this information, and fetches the string "ABCDEF" to pass on to Task B.

So in terms of memory usage, what do you have? You have one location where the string is stored in memory, a simple look up table stored in memory that is controlled by the MAU, the MAU itself which provides this abstraction layer and offers opportunities to control memory usage, and the memory spaces used by the tasks.

The address 0x0FF can be called as many times as you want (please note this virtual memory value can only be assigned to one real memory space at a time), but each time the MAU will make a decision based on certain rules to determine whether that access is given. It will also be giving programs non overlapping memory spaces to run in.

Does this explanation help?

EDIT: Should also say that I intend to separate kernel commands from the kernel itself so that its commands go through the MAU too (obviously with a high level of privileges). I thought that was worth pointing out in case you thought I was intending to do MP with the standard design Amiga kernel.

[HenryCase]

@all

I am sorry for the mess I instigated in this thread. I will be making further MP comments in the designated thread, and I recommend everyone else does the same. You can still quote from this thread if you wish.

Back to the Coldfire discussion. I think we should be looking to get help with implementing full 68k compatibility on Coldfire. FrenchShark obviously has skills in this area, and I'm sure Oli-HD has useful knowledge too. Thomas and Gunnar are busy enough without adding more to their workload. I've been looking at projects on the Internet where people have tried to move operating systems from 68k to Coldfire. The two projects I've found so far are:

Debian/68k
http://www.nabble.com/debian-68k-f12565.html

Atari Coldfire Project
http://acp.atari.org/

Would anyone object to me attempting to make contact and getting others involved with CFNatami?