How to move AROS forward

[HenryCase]

@bloodline
I'm glad we're getting somewhere now, you were right I was unaware of the pointer usage in AmigaOS.

bloodline wrote:
As an example,

Two task in AmigaOS... A and B

Task A will allocate some memory and fill it with some Data, and then send a pointer to Task B.  Task B receives that pointer and uses it to access the Data put there by Task A. This is a very quick process and requires only the memory allocated by Task A.

Then what we would need for MP would be a way of sticking the OS in-between all communications between tasks by manipulating the pointer data. First we need a way of flagging up when a pointer has been set by a program. I've used this guide to give me an overview of how pointers work:
http://www.codeproject.com/KB/cpp/pointers.aspx
So in C you set the pointer variable with a command like this:
int* pNumberOne; (for a pointer of integer type)

Now going to compare this with Pascal. Using this guide...
http://www.learn-programming.za.net/programming_pascal_learn13.html
So if you wanted to declare the same pointer in Pascal you would use these commands:
var
   pNumberOne: ^integer;

My question is this. When it comes to compiling the C and Pascal programs will they have the same machine code value for the pointer we declared?

[Piru]

@HenryCase

You can't in any sensible way intercept the pointer usage, so the question is irrelevant. You can't "flag" or any other way know when the app is using pointers and/or if it's using pointers correctly or not.

You can't have MP with AmigaOS, get over it.

The best limited MP hack for AmigaOS is MMULib by Thomas Richter. It has a MuLink tool that can write protect certain segments of the executables. Obviously the program in question must be compiled to take advantage of this (basically only code and read only data can be protected etc). This is far from full MP however.

[A1260]

this is really easy, focus on the important x86 version of aros, after all it was there it all started. the rest of the aros ports is of no important at the moment. finish x86 aros with everything that is necessary, like an amiga emulator integrated for classic games and software and firefox web browser...

[HenryCase]

Piru wrote:
@HenryCase

You can't in any sensible way intercept the pointer usage, so the question is irrelevant. You can't "flag" or any other way know when the app is using pointers and/or if it's using pointers correctly or not.

No sensible way intercept pointer usage? This is linked to the 'can it be done elegantly' question I mentioned previously, whereas this discussion is still at the 'can it be done' stage. Even if the MP solution we find is impractical, but works, then at least we'll know what our real options are.

So I'll ask again: is it possible to write a program that can scan the binary of another application and know when a pointer has been set?

[bloodline]

HenryCase wrote:
@bloodline
I'm glad we're getting somewhere now, you were right I was unaware of the pointer usage in AmigaOS.

bloodline wrote:
As an example,

Two task in AmigaOS... A and B

Task A will allocate some memory and fill it with some Data, and then send a pointer to Task B.  Task B receives that pointer and uses it to access the Data put there by Task A. This is a very quick process and requires only the memory allocated by Task A.

Then what we would need for MP would be a way of sticking the OS in-between all communications between tasks by manipulating the pointer data. First we need a way of flagging up when a pointer has been set by a program.

There is no way to know what the programs are passing to each other. If Task A sends a number to Task B, the OS has no way of knowing how Task B will use the number! That number could be a pointer, or it could be some data... there is no difference technically, to the OS and CPU it is just a number.

AmigaOS simply does not have any mechanism, and worse, never has had any provision for identifying what is being passed between programs.

I've used this guide to give me an overview of how pointers work:
http://www.codeproject.com/KB/cpp/pointers.aspx
So in C you set the pointer variable with a command like this:
int* pNumberOne; (for a pointer of integer type)

Now going to compare this with Pascal. Using this guide...
http://www.learn-programming.za.net/programming_pascal_learn13.html
So if you wanted to declare the same pointer in Pascal you would use these commands:
var
   pNumberOne: ^integer;

My question is this. When it comes to compiling the C and Pascal programs will they have the same machine code value for the pointer we declared?

As I keep saying in every post, you are thinking high level... the Computer does not work at this high level... there is no difference at a machine level between a number and a pointer!

[Piru]

@HenryCase

So I'll ask again: is it possible to write a program that can scan the binary of another application and know when a pointer has been set?

And I answer again: No.

[bloodline]

HenryCase wrote:

Piru wrote:
@HenryCase

You can't in any sensible way intercept the pointer usage, so the question is irrelevant. You can't "flag" or any other way know when the app is using pointers and/or if it's using pointers correctly or not.

No sensible way intercept pointer usage?

No way, period.  You could make assumptions but at best you would only be right, less than 30% of the time.

This is linked to the 'can it be done elegantly' question I mentioned previously, whereas this discussion is still at the 'can it be done' stage. Even if the MP solution we find is impractical, but works, then at least we'll know what our real options are.

It can't be done... No system could be made that would be in anyway deterministic.

So I'll ask again: is it possible to write a program that can scan the binary of another application and know when a pointer has been set?

No. There is nothing special about a pointer... it looks exactly the same as any other kind of number.

[HenryCase]

@bloodline and @Piru
Thank you for your posts, I will now reply to them in one go.

bloodline wrote:
There is no way to know what the programs are passing to each other.

One thing at a time. First we need to know where the pointers are, then I'll say about my ideas on how we can use this information.

Piru wrote:
And I answer again: No.

Please explain the reasoning behind your answer, a simple 'No' doesn't help me understand why you don't think there is a way to see pointers in binary code.

Whilst I wait for your explanation, I would like to ask you another question. How does a CPU know that a program is asking to set up a pointer?

bloodline wrote:
No way, period. You could make assumptions but at best you would only be right, less than 30% of the time.

ASM is closely linked to the structure of machine code, correct? Also, ASM strives for efficiency. I would argue that there are only so many ways to set a pointer in ASM efficiently, and therefore only so many binary structures we would need to scan for to get a higher accuracy than 30% (where you got that figure from I don't know). I found a couple of ways of setting pointers in x86 ASM (I'm assuming it's x86 ASM anyway) from this page (Google cache version):
cs.colgate.edu/faculty/nevison/cs201web/lectureNotes/arraysandpointers.ppt
========================
clear2:
add  $t0, $a0, $zero    # p = addr of array[0]
forp:   sw   $zero, 0($t0)      # memory[p] = 0
addi $t0, $t0, 4        # p = p + 4
sll  $t1, $a1, 2        # $t1 = 4 * size
add  $t1, $t1, $a0      # $t1 = addr  array[size]
slt  $t2, $t0, $t1      # p < &array[size] ?
bne  $t2, $zero, forp   # if so, continue
endfor:
=========================
Pointer version assembly code
clear2:
add  $t0, $a0, $zero    # p = addr of array[0]
sll  $t1, $a1, 2        # $t1 = 4 * size
add  $t1, $t1, $a0      # $t1 = addr  array[size]
forp:   sw   $zero, 0($t0)      # memory[p] = 0
addi $t0, $t0, 4        # p = p + 4
slt  $t2, $t0, $t1      # p < &array[size] ?
bne  $t2, $zero, forp   # if so, continue
endfor:
========================
Why is it impossible to scan for the binary versions of the ASM code above?

[bloodline]

HenryCase wrote:
@bloodline and @Piru
Thank you for your posts, I will now reply to them in one go.

bloodline wrote:
There is no way to know what the programs are passing to each other.

One thing at a time. First we need to know where the pointers are, then I'll say about my ideas on how we can use this information.

You can't say that!

What if the pointer never actually exists anywhere and is calculated by Task A and then passed to Task B?

Piru wrote:
And I answer again: No.

Please explain the reasoning behind your answer, a simple 'No' doesn't help me understand why you don't think there is a way to see pointers in binary code.

I'll answer, again, since I've repeatedly said it before :-)

There is nothing special about a pointer... it is just a number, neither the CPU or the OS has any idea what that number is for.

The CPU just pushes numbers around... buy a good book on 68K ASM and learn the CPU inside out... it shouldn't take more than a few weeks to figure it out... you can't hope to understand this problem until you know a CPU.

Whilst I wait for your explanation, I would like to ask you another question. How does a CPU know that a program is asking to set up a pointer?

There is no "setting up a pointer", it's just a number to the CPU... nothing special.

bloodline wrote:
No way, period. You could make assumptions but at best you would only be right, less than 30% of the time.

ASM is closely linked to the structure of machine code, correct?

ASM is the human readable form of machine code, but often has some high level features to aid coding.

Also, ASM strives for efficiency.

Not really... My ASM code was far from efficient ;-)

 I would argue that there are only so many ways to set a pointer in ASM efficiently, and therefore only so many binary structures we would need to scan for to get a higher accuracy than 30% (where you got that figure from I don't know).

My 30% figure was if you tried to assume what the tasks were sending to each other.

It really doesn't matter what a task does in it's own address space. The problems occur when other tasks attempt to access structures which are not in their address space.

Please get over this obsession with pointers... they are not going to solve any problems.

The problems are with the design of AmigaOS... not the CPU or the programming languages used!!

 I found a couple of ways of setting pointers in x86 ASM (I'm assuming it's x86 ASM anyway) from this page (Google cache version):
cs.colgate.edu/faculty/nevison/cs201web/lectureNotes/arraysandpointers.ppt
========================
clear2:
add  $t0, $a0, $zero    # p = addr of array[0]
forp:   sw   $zero, 0($t0)      # memory[p] = 0
addi $t0, $t0, 4        # p = p + 4
sll  $t1, $a1, 2        # $t1 = 4 * size
add  $t1, $t1, $a0      # $t1 = addr  array[size]
slt  $t2, $t0, $t1      # p < &array[size] ?
bne  $t2, $zero, forp   # if so, continue
endfor:
=========================
Pointer version assembly code
clear2:
add  $t0, $a0, $zero    # p = addr of array[0]
sll  $t1, $a1, 2        # $t1 = 4 * size
add  $t1, $t1, $a0      # $t1 = addr  array[size]
forp:   sw   $zero, 0($t0)      # memory[p] = 0
addi $t0, $t0, 4        # p = p + 4
slt  $t2, $t0, $t1      # p < &array[size] ?
bne  $t2, $zero, forp   # if so, continue
endfor:
========================
Why is it impossible to scan for the binary versions of the ASM code above?

This looks like PPC asm... which I really don't like and was never meant for humans to read... but you miss the point, the CPU doesn't see the high level part of the code. it just adds... subtracts... shifts... ands... ors... nots... numbers! It really doesn't have any idea what the programmer is trying to do, it can't know, and nothing outside of the CPU can know either!

What you are presenting here is totally irrelevant to the problem, which is actually very simple. AmigaOS doesn't know what programs are saying to each other or how programs interact with each other.... AmigaOS allows the programmer to make the programs work in anyway he chooses, that is a design feature of AmigaOS to save memory and be quick on a 7Mhz CPU... Remove that feature and nothing written for AmigaOS works anymore.

Once you add MP, you have to impose restrictions on the programmer and provide him with clearly defined interfaces and methods for achiving a task.

-Edit- I want to add that the pointer problem is deeper than just sharing Data... Code is also shared on AmigaOS, the entire library system is built around the idea than one executable and jump into and run code in another executable... something totally alien to a Memory protected OS...

[SamuraiCrow]

@HenryCase

To scan for those binary equivalents you'd have to know the difference between code and data.  If you write a patcher to go through and change all of the binary instances of memory access outside the main chunk of code, you'd probably end up inadvertently changing some constant data to something different.  In short you'd have to go into every program by hand and write patch code for every instance of pointer access.

Why not just get the source codes of everything you'd ever want to run on AROS and recompile it for MacOSX?  You'd save yourself a lot of headaches and the rest of us a lot of hassle. :-x

[XDelusion]

I could go for a port of PrBOOM and zDOOM (if not GzDOOM) while we are waiting on updates.

[bloodline]

SamuraiCrow wrote:
@HenryCase

To scan for those binary equivalents you'd have to know the difference between code and data.  If you write a patcher to go through and change all of the binary instances of memory access outside the main chunk of code, you'd probably end up inadvertently changing some constant data to something different.

I would actually be surprised if you even could identify the memory access, you have no way to know what and where that pointer is going... thus why a JIT can only translate code blocks up to the jumps.

In short you'd have to go into every program by hand and write patch code for every instance of pointer access.

And even if you could do that, it doesn't solve the problem... AmigaOS programs are designed at a fundamental level to interact intimately, you would have to rewrite the program completely.

Why not just get the source codes of everything you'd ever want to run on AROS and recompile it for MacOSX?  You'd save yourself a lot of headaches and the rest of us a lot of hassle. :-x

I agree... but you'd have a hell of a time porting a program from AmigaOS to MacOSX... I would rather just rewrite the program.

[bloodline]

XDelusion wrote:
I could go for a port of PrBOOM and zDOOM (if not GzDOOM) while we are waiting on updates.

AROS has both Doom and Quake ports, that work really well... :-)

[paolone]

bloodline wrote:

AROS has both Doom and Quake ports, that work really well... :-)

And it would be really fine if someone would look at their sources, and add AHI audio support.

[Hans_]

@HenryCase

The OS4 development team have looked at this, and decided that the only way to make it happen would be to break compatibility. Piru and Bloodline say so too. Trying to somehow discover who owns what memory, and who's still using it would be way to costly computationally, even if you could do it. The correct way forward is to create stricter rules about sharing memory, and that can't be done without breaking compatibility.

Whether it's AOS4.x, MOS, or AROS, the only way to get full memory protection would be to modify the API in a manner that breaks compatibility. Old apps would be run in a sand-boxed environment.

I'd be happy if they did break compatibility and provide a sand-box for old apps. While they're doing that, they might as well break other stuff to allow SMP, and to lock off user programs from accessing OS structures.

Hans

[ffastback]

paolone wrote:

1. just whining and claiming their opinion would be the best way to improve AROS

2. contribute with code to reach the common goal stated above.

3. take the existing code and improve upon it the way they like, just to become the new ruler of the world.

The direction is already here for the people that want to see it. The others may disagree, but please don't claim AROS is pointless or directionless anymore, as it is completely untrue.

How about adding:

4.  If you can't program, donate to an AROS bounty, even if $1.