Welcome, Guest. Please login or register.

Author Topic: How to move AROS forward  (Read 30411 times)

Description:

0 Members and 3 Guests are viewing this topic.

Offline bloodline

  • Master Sock Abuser
  • Hero Member
  • *****
  • Join Date: Mar 2002
  • Posts: 12113
    • Show all replies
    • http://www.troubled-mind.com
Re: How to move AROS forward
« on: July 25, 2008, 12:09:52 AM »
Quote

uncharted wrote:


If I think a project's overall direction is flawed, am I going to waste time submitting code that won't change that?  Would that really be a good use of my time?  Or are you suggesting that I do the whole lot myself?



There is no direction!  The only goal is to get something that works, and each dev works to improve what they can, what they know how to improve...

I appreciate you don't have much respect for the project, I'm quite sure it offers you nothing... but if you compile the latest sources, you end up with a usable OS, that is in every way like AmigaOS... with a bunch of programs to play with... frankly that's pretty much all my Amigas can do too... I wouldn't get much more from any other Amiga system...

I would like a 68k build, and there are people messing around with that right now... but no Amiga system can really offer any more than than what AROS offers now... so I am happy.

Offline bloodline

  • Master Sock Abuser
  • Hero Member
  • *****
  • Join Date: Mar 2002
  • Posts: 12113
    • Show all replies
    • http://www.troubled-mind.com
Re: How to move AROS forward
« Reply #1 on: July 25, 2008, 11:15:36 AM »
Quote

HenryCase wrote:
Quote
Matt_H wrote:
With the current lack of manpower, formally forking AROS will just create 2 stagnant projects.


Bingo.


Hahaha, but no... Go on fork it now, it's easy! Download the souce code, set up an SVN server, and then develop it how ever you like... Try and get devs on board though and see how far you get... If you make a sucess then what have we lost?

But like most in the Amiga scene it's all just talk... at least the AROS team as it is DOES achieve something.

I summise that the current AROS devs just work in their spare time and work on what parts they understand or find interesting. They do a fine job. If you make fork, they're not going to work on that fork (they don't have the time or the motivation)... the Main AROS tree wil develop as it always has, and your fork will die.

Quote

Quote
saimon69 wrote:
Inside the same thread a proposal from HenryCase in order to avoid a fork and provide both new ways to evolve AROS (such as the holy memory protection grail) and in the same time provide the compatibility with the 3.1 API, but cannot say more about it because he haven't laid out anywhere yet, well maybe this is the right time....


Well remembered. I'm not going to say any more on the subject until I'm ready to code a 'proof' for my theory (the MP issue always causes a sh*tstorm), so you're right, now is not the time.


It can't be done... the AmigaOS design does not allow for MP... I am happy to talk you through it again... it's a topic I actually enjoy talking about.

Quote

Quote
uncharted wrote:
Proposing a sensible path forward (see above - you did actually read it all before going onto the defensive didn't you?)


uncharted, thank you for ideas, but as you can see from saimon69's post AROS fans already discussed these same ideas a few months ago. Some would argue we lost a good developer out of those discussions (though there were other factors at play too).


Yes, we lost a great developer... but even though he has now left, he contributed a great deal to the AROS project!

Quote

I'd say yakumo9275's response was justified. Too many armchair experts (including myself), not enough developers, that's AROS's main problem, so what are you going to do to fix that?


I don't think AROS actually has a problem at all... we are half way throguh 2008... the whole concept of the Amiga is VERY VERY dead.

AROS is now very much a hobby project, and as such it's actually doing very well. I can install it on my various computers, it functions just as my old Amigas function, it's fun to play with... it costs me nothing to mess around with... I have the source code if I want to try out an operating system idea, I can try it out, because of AROS I understand how AmigaOS actually works and the design choices made and why... and seeing the flaws in it made me apprciate the design choices and trade offs made in other operating systems...

Quote

Anyway, contribute something concrete to AROS and I'm sure you'll get a more favourable response to your ideas.


Anyone is free to contribute to AROS, and also use AROS as a learning tool!

Offline bloodline

  • Master Sock Abuser
  • Hero Member
  • *****
  • Join Date: Mar 2002
  • Posts: 12113
    • Show all replies
    • http://www.troubled-mind.com
Re: How to move AROS forward
« Reply #2 on: July 26, 2008, 02:47:51 PM »
Henry...

There is only one small conceptual point that you are missing. I have been racking my brains to try and find a good metaphor for the problem... bur I can't really find one.

I want to address your conceptual error as you raised it before... That is this strange idea of your that a program requests data from memory. It doesn't.

What we have is a big flat area of memory, and a CPU.
The CPU does not know or care where it is reading and writing, it simply follows what it is told to do by a computer program. The CPU in AmigaOS has no information regarding the status of the code at all. While AmigaOS offers contexts (where each exec task has it's own set of certain data structures etc), these contexts are all part of the same address space.

In the AmigaOS design, the designers decided that they wanted the system to be fast and use as little memory as possible. Thus AmigaOS was designed with the idea that any data structure that could be shared, would be shared... No provision was made in the design of Amiga to control the access to system wide data structures... this was a design decision, and it can't be retro fitted without totally redesigning the system, and it would no longer be AmigaOS compatible.

You are thinking high level, not machine level... Your ideas are actually an area of research called "managed code", the most famous is MicroSoft's Singularity project. There programs access memory via defined system functions thus memory access can be controlled without the need for an MMU.

But AmigaOS is not a managed code system, it was designed as an operating system to be programmed at a machine code level... in fact I can write a program in ASM that would work on AmigaOS in a matter of minutes... I certainly couldn't even begin to think about doing that on a real operating  like a UNIX or Windows... where I would have to stick with C.

Offline bloodline

  • Master Sock Abuser
  • Hero Member
  • *****
  • Join Date: Mar 2002
  • Posts: 12113
    • Show all replies
    • http://www.troubled-mind.com
Re: How to move AROS forward
« Reply #3 on: July 26, 2008, 03:05:24 PM »
Quote

HenryCase wrote:
Quote
itix wrote:
@HenryCase
One problem in memory protection is that applications and the operating system are sharing system structures. For example when you call OpenWindow() the system returns struct Window which is globally shared. If your application trashes this structure it may lead to crash of operating system.


Thanks for your reply itix. I looked into the OpenWindow() API call and I can't see anything that would prevent changing it from a global variable to a more local one.


But the data structures are shared... Every program ever written for AmigaOS and the System itself expect to have access to these data structures... what do you want to do? Give each one it's own local copy of intuition? How the hell could that work? The way intuition was designed doesn't allow for that!

Quote

Quote
itix wrote:
The design of Amiga is unfortunately flawed for memory protection. One could say that on Amiga there is no user space and kernel space -- there is only kernel space where all applications are running.


This is the case now, but we're looking at ways to create a user space-kernel space separation whilst keeping OS3.1 API compatibility.


There is no separation in the design of the API... the actual concept of the OS is everything running in Kernel Space... This isn't a push a button and the Kernel is in a separate space problem... there is no separation in the API design!

Quote

 This is not your daddy's exec if you catch my drift.


No, you really need to spell this out because it is currently a hidden component in your theory and possibly one that is causing your confusion.

Quote

 If it is possible to implement a new kernel with MP that supports the OS3.1 API calls then we can have MP in an Amiga based OS.


But those very same API calls return pointers to shared data structures... how do you get around that?

Quote

So a good question is this: are there any global variables that cannot be contained more locally?


That's an irrelevant question, you are thinking as if the CPU is running high level code, it is not.

Offline bloodline

  • Master Sock Abuser
  • Hero Member
  • *****
  • Join Date: Mar 2002
  • Posts: 12113
    • Show all replies
    • http://www.troubled-mind.com
Re: How to move AROS forward
« Reply #4 on: July 26, 2008, 04:18:46 PM »
Quote

HenryCase wrote:
@bloodline
Thanks for your response.

Quote
bloodline wrote:
What we have is a big flat area of memory, and a CPU.
The CPU does not know or care where it is reading and writing, it simply follows what it is told to do by a computer program. The CPU in AmigaOS has no information regarding the status of the code at all. While AmigaOS offers contexts (where each exec task has it's own set of certain data structures etc), these contexts are all part of the same address space.


I agree, the CPU doesn't have to care where it is reading or writing, it just has to follows what it is told to do by a computer program, namely the new AROS memory manager I am proposing. AROS is software too. I'm taking control over memory from the applications and giving it to the OS. If a program uses an API call like AllocMem() then AROS knows about it, right?


But in AmigaOS there is no way to control what a program does with that memory... It could quite happily pass the pointer returned by AllocMem() to another program... and that is a system legal thing to do in AmigaOS, this is a fundamental concept in AmigaOS that allow the system to function... if you have memory protection the pointer that the program received from the AllocMem() function doesn't exist anywhere but in that program's address space... so as soon as it tries to pass that pointer to another program/system function/etc it has just passed bad Data!! There is NO way in AmigaOS to know what a program is passing to another program.

Quote

Quote
bloodline wrote:
In the AmigaOS design, the designers decided that they wanted the system to be fast and use as little memory as possible. Thus AmigaOS was designed with the idea that any data structure that could be shared, would be shared... No provision was made in the design of Amiga to control the access to system wide data structures... this was a design decision, and it can't be retro fitted without totally redesigning the system, and it would no longer be AmigaOS compatible.


Clearly I'm not aiming to clone every design decision made by the AmigaOS designers.


Then you won't have AmigaOS... it would be like adding Gendered and Cased nouns to English... no existing English speaker would be able to use it... as it wouldn't be English anymore.

Quote

I'm trying to focus on how we keeping OS 3.1 API compatibility in a MP system, we can discuss if the resultant OS is Amiga-like later.


Amiga-like = AmigaOS 3.1 API compatibility. By the AROS definition.

Quote

Quote
bloodline wrote:
You are thinking high level, not machine level... Your ideas are actually an area of research called "managed code", the most famous is MicroSoft's Singularity project. There programs access memory via defined system functions thus memory access can be controlled without the need for an MMU.


Interesting. Can you give me an overview of how Microsoft's Singularity achieves this feat?


I already did... The concept of the memory pointer doesn't exist in a managed code system. All memory access is mediated by the Operating system. it's a slow way to operate, but it does allow stability on non MMU systems. The overhead of such a system make it unsuitable for real use... thus it was just a research project, now abandoned by MicroSoft, and released to the Open Source community.

Google and wikipedia are your friends here... I don't have to hold you hand the whole way?

Quote

Quote
bloodline wrote:
But AmigaOS is not a managed code system, it was designed as an operating system to be programmed at a machine code level... in fact I can write a program in ASM that would work on AmigaOS in a matter of minutes... I certainly couldn't even begin to think about doing that on a real operating like a UNIX or Windows... where I would have to stick with C.


AmigaOS may not have been a managed code system historically, but we're not looking at what it was but rather what we can make it without breaking compatibility.


what you are suggesting is the same as me saying... I shall try and cook a Chicken in a cement mixer... Just not possible... you would end up with mashed raw chicken...

Quote

You'd still be able to code in ASM with my MP system. The programs making AmigaOS API calls see the 'virtual' memory layout, but the users of the OS can see the 'virtual' memory layout and the 'real' memory layout. I hope you can see that gives us options to allow ASM code to run alongside other code.


Oops, I've confused you with my ASM example... I was simply trying to show how Simple the design of AmigaOS was... not that ASM was good or bad... One can easily write an computer program on Any OS in ASM (except a managed code system, were everything must be high level), but it's harder work because they are more complex.

Quote

One option is to add a new API call(s) especially for ASM code.


And what program is going to use a function call it doesn't know about!?!?

Quote

When the ASM program is executed it first makes an API call that says something like 'I will need memory from XXXX to XXXX to run'. If this memory is free then it is assigned to the ASM program. If it is not free you can either:


What? Why? This really makes no sense at all?!?

Quote

1. Close the programs using that memory.


??? Why... What if that program was your Email client?

Quote

2. Move the programs using the memory to another memory space (the programs are paused during this move, but can carry on afterwards as normal due to the 'virtual' memory addresses not moving.


How the hell do you move it? remember AmigaOS has no idea what program is using what memory...

You really must read about how virtual memory works as, this statement shows that you don't understand it at all!

Quote

3. A copy of the ASM program is made, and this copy is amended with available memory address locations before being run (the application doing this works like a find and replace function in a word processor).


Again, you haven't made any sense here...

Quote

However, ASM by its nature does not allow MP(AFAIK),


ASM has nothing to do with MP... that is a design consideration of the OS, not the programing language you have used.

Quote

unless you sandbox it, so yes you could crash AmigaOS with a badly coded ASM app, but the same is true of any operating system.


No it isn't... In a real OS... like OSX, no program can take down the OS... no matter how you wrote it. that is the point of MP.

Quote

If you code ASM apps so that they stay out of the way of the area where the OS is running from you should be okay.


That is the point of MP... so if a program tries to hit the OS space... it can't, this is done by sitting a program in it's own address space, where the OS does not exist... the App has no ability to see any other memory than what is in it's own address space... once the CPU is in that task's context it cannot access (read or write) outside of that context's address space.

Offline bloodline

  • Master Sock Abuser
  • Hero Member
  • *****
  • Join Date: Mar 2002
  • Posts: 12113
    • Show all replies
    • http://www.troubled-mind.com
Re: How to move AROS forward
« Reply #5 on: July 26, 2008, 04:27:12 PM »
Quote

HenryCase wrote:
@bloodline
I'll reply to your last post shortly, outlining my ideas for the new exec will take a little time.

@Hans
Quote
Hans_ wrote:
Anything running in supervisor mode should have full access to all memory in the system. Thus, the OS kernel wouldn't have to change ownership back and forth. For non-kernel OS modules that run in user mode, they could run directly in the application's task/process; this means that there still wouldn't be any ownership change.


Correct. The memory in use would be jointly owned by the kernel and the applications permitted to run in that space. This allocation only changes when you close an application or give a new application rights to use that space.


No, Hans_ is confusing the issue.

If you allow a system structure to be shared between apps, then it only takes one app to to accidentally or maliciously destroy that data structure to take that component down and the system is now broken...

But shared memory areas are not a problem, and most OSs do allow a shared memory space, but their use is very carefully defined in the OS design, and nothing mission critical can be put there.
 
With AmigaOS, you have NO way to monitor what is being passed between programs/system functions... this is a design feature of AmigaOS... it is defined by the way AmigaOS works!

As soon as you add memory protection, the programs and functions will be passing rubbish between each other... and there is nothing you can do about that, because of the design of AmigaOS!!!

If you add MP, you will be altering so much of the system that you will end up with a new operating system that is totally incompatible with AmigaOS 3.1... and if you are going to that much trouble, why not just take an existing system that already has the design features you want...

In 1998 Apple had the same problem, MacOS was not able to have MP or multitasking by design... 6 years of attempting to retrofit them was fruitless... so they brought in a new system and then put an Emulator to run the old Apps... that would have been the only solution for AmigaOS too.

Offline bloodline

  • Master Sock Abuser
  • Hero Member
  • *****
  • Join Date: Mar 2002
  • Posts: 12113
    • Show all replies
    • http://www.troubled-mind.com
Re: How to move AROS forward
« Reply #6 on: July 26, 2008, 07:52:12 PM »
Quote

HenryCase wrote:
@bloodline
Whilst thinking about how we would need to change the core of AmigaOS to allow for MP, I thought of a talking point we can use to (hopefully) progress our discussion into something more productive. Am I right in thinking that your main argument behind 'no MP' with the OS3.1 API is because you can't think of a way of adding resource tracking without breaking compatibility?


No, not at all. AROS in fact implements resource tracking.

Quote

If I am correct, then lets look at how API calls with MP and non-MP operating systems compare. Lets take the allocation of memory API call for example. In AmigaOS we have AllocMem:

http://utilitybase.com/ref/?keyword=AllocMem&funcgroup=AmigaOS&action=Search
"memoryBlock = AllocMem(byteSize, attributes)"

Now let's look at malloc, the equivalent memory allocation function in FreeBSD, an OS with MP:
http://www.freebsd.org/cgi/man.cgi?query=malloc&sektion=9
"malloc(unsigned long size, struct malloc_type *type, int flags);"

Okay, so let's compare these API calls. First we can see they both have attributes that allow you to set the size of memory space required. They also both have flags/attributes you can set to determine the behaviour of that memory space. With AllocMem (AmigaOS), we have the following flags:

MEMF_CHIP (request ChipMem), MEMF_FAST (request FastMem), MEMF_LOCAL (memory survives CPU reset), MEMF_24BITDMA (used for setting memory for Zorro devices), MEMF_KICK (allows memory to be accessed by Exec early in boot process), MEMF_CLEAR (sets memory to all 'zeros'), MEMF_REVERSE (allocate from top of memory pool), MEMF_NO_EXPUNGE (prevents memory from being expunged on failed memory allocation)

Now let's look at the flags we can set using malloc (FreeBSD):

M_ZERO (sets memory to all 'zeros'), M_NOWAIT (returns value if memory cannot be set immediately), M_WAITOK (indicates okay to wait for memory to be set), M_USE_RESERVE (indicates memory can be set in memory reserved for OS use)

Take a good look at AllocMem and malloc. If I didn't tell you that one function was used on a non-MP OS and the other was on a MP OS, would you have been able to tell just by looking at the API call structure? No.

Similarly, the API call itself doesn't care if it's running on a MP OS or non-MP OS. There is no 'anti-MP' flag for AllocMem. Resource tracking is a separate issue from API calls, it is what the kernel handles.

Thoughts?


Ok, I can see where you are missing the problem I think.

The problem is not that you can't add MP to AmigaOS. The problem is that when you add it, the system doesn't work like AmigaOS any more.

In an MP OS, you pass data between programs and system functions. This is how they communicate. This is clearly defined behavior, as the OS and the programs "know" that they all live in separate address spaces... which are to all intents and purposes separate computers.

In AmigaOS, programs pass pointers, NOT DATA, to each other... these pointers reference memory anywhere in the flat address space.  As soon as each task gets it's own address space, the pointers which are passed between programs are meaningless!!! Thus all amiga software stops working! There is no way to fix this since AmigaOS only has one message passing system and the OS puts no controls on what is passed.

Offline bloodline

  • Master Sock Abuser
  • Hero Member
  • *****
  • Join Date: Mar 2002
  • Posts: 12113
    • Show all replies
    • http://www.troubled-mind.com
Re: How to move AROS forward
« Reply #7 on: July 26, 2008, 08:50:44 PM »
As an example,

Two task in AmigaOS... A and B

Task A will allocate some memory and fill it with some Data, and then send a pointer to Task B.  Task B receives that pointer and uses it to access the Data put there by Task A. This is a very quick process and requires only the memory allocated by Task A.

In an MP OS there are several strategies used to pass data, I will only present a simple one here.

Task A will allocate some memory and fill it with Data. It then must actually send that Data to task B, this means physically copying the Data from Task A's address space to Task B's Address space byte by byte...
there are actually other strategies that one might employ (since this is rather slow) but it shows clearly the problem which we have to overcome!
Task B must have it's own copy of the same Data in it's own address space in order to work on it.

Amiga OS can happily use the slower more complex strategies for passing data, which are MP compatible... but these methods were never required and thus not used either by the OS or any applications... so we are stuck with a non MP compatible OS... since there is no way to know what the OS and task are passing to each other!

Offline bloodline

  • Master Sock Abuser
  • Hero Member
  • *****
  • Join Date: Mar 2002
  • Posts: 12113
    • Show all replies
    • http://www.troubled-mind.com
Re: How to move AROS forward
« Reply #8 on: July 26, 2008, 08:59:01 PM »
In your AllocMem() Vs malloc() example...

The difference is this;

AllocMem() returns an address that points to the same bit of memory to any program that were to receive that pointer.

malloc() returns an address that is meaningful only to the task that called the function... any other task that receives that pointer has just received a meaningless pointer which points to nowhere that it can address!

Offline bloodline

  • Master Sock Abuser
  • Hero Member
  • *****
  • Join Date: Mar 2002
  • Posts: 12113
    • Show all replies
    • http://www.troubled-mind.com
Re: How to move AROS forward
« Reply #9 on: July 26, 2008, 11:18:27 PM »
Quote

Leo24 wrote:
Hi guys,

sorry I didn`t read the whole thread...but here some notes from me. I suppose MP in this context means Multiprocessing, right?


No, we are talking about Memory Protection.

Lets not even go near Multiprocessing yet as that is a whole other issue, which AmigaOS isn't well suited to...

Quote

It`s true that most comercial and free operating systems out there use a separate address space for each process. This is a design decision** and not mandatory nor necessary for a MP capable OS. It`s possible to have MP working on a single address space and to use the MMU to extend this space beyond the physical memory by using the page swaping techique and to handle memory protection. Shared resources must still be protected through mutexes or semaphores from concurrent CPU accesses. In a single (flat) address space OS (SASOS) message passing is done by passing pointers to the data in memory, just like AmigaOS does, there is no need to move or copy data around. Note that even in non-SASOS data is generally not copied around, it`s mapped with the help of the MMU, so both processes may see the same data in the same place or in case a process already allocated that virtual memory region for other stuff, in different memory places.


AmigaOS tasks often rely on a Forbid() halting all multitasking and that a high priority task WILL block a lower priority one... on a Multiprocessor system this condition may not always be true.

Quote

** in the early days of computing when there was 16 bit address spaces only, address translations didn`t make much sense because one could not get over the 16 bit address space with it. Earlier MMUs just swapped memory regions with the bank switching method, thus having many code blocks using the same adresses but only one at a time. In the 32 bit era this technique became obsolete and segmenting and paging took over. However, using for each process different address spaces mapped to the same region simplified program loading as programs could be linked statically and did not need to be relocated in memory. In the 64 bit era there is realy no need to have many address spaces any more. Newer research OSes (e.g. singularity from Microsoft) are in fact SASOS.

Best regards,
Leo24

Offline bloodline

  • Master Sock Abuser
  • Hero Member
  • *****
  • Join Date: Mar 2002
  • Posts: 12113
    • Show all replies
    • http://www.troubled-mind.com
Re: How to move AROS forward
« Reply #10 on: July 27, 2008, 12:35:37 AM »
Quote

itix wrote:
@bloodline

Quote

No, not at all. AROS in fact implements resource tracking.


Does it also work? :-)


Not really... :-D

Offline bloodline

  • Master Sock Abuser
  • Hero Member
  • *****
  • Join Date: Mar 2002
  • Posts: 12113
    • Show all replies
    • http://www.troubled-mind.com
Re: How to move AROS forward
« Reply #11 on: July 27, 2008, 07:27:11 PM »
Quote

HenryCase wrote:
@bloodline
I'm glad we're getting somewhere now, you were right I was unaware of the pointer usage in AmigaOS.

Quote
bloodline wrote:
As an example,

Two task in AmigaOS... A and B

Task A will allocate some memory and fill it with some Data, and then send a pointer to Task B.  Task B receives that pointer and uses it to access the Data put there by Task A. This is a very quick process and requires only the memory allocated by Task A.


Then what we would need for MP would be a way of sticking the OS in-between all communications between tasks by manipulating the pointer data. First we need a way of flagging up when a pointer has been set by a program.


There is no way to know what the programs are passing to each other. If Task A sends a number to Task B, the OS has no way of knowing how Task B will use the number! That number could be a pointer, or it could be some data... there is no difference technically, to the OS and CPU it is just a number.

AmigaOS simply does not have any mechanism, and worse, never has had any provision for identifying what is being passed between programs.

Quote

I've used this guide to give me an overview of how pointers work:
http://www.codeproject.com/KB/cpp/pointers.aspx
So in C you set the pointer variable with a command like this:
int* pNumberOne; (for a pointer of integer type)

Now going to compare this with Pascal. Using this guide...
http://www.learn-programming.za.net/programming_pascal_learn13.html
So if you wanted to declare the same pointer in Pascal you would use these commands:
var
   pNumberOne: ^integer;

My question is this. When it comes to compiling the C and Pascal programs will they have the same machine code value for the pointer we declared?


As I keep saying in every post, you are thinking high level... the Computer does not work at this high level... there is no difference at a machine level between a number and a pointer!

Offline bloodline

  • Master Sock Abuser
  • Hero Member
  • *****
  • Join Date: Mar 2002
  • Posts: 12113
    • Show all replies
    • http://www.troubled-mind.com
Re: How to move AROS forward
« Reply #12 on: July 27, 2008, 07:31:07 PM »
Quote

HenryCase wrote:
Quote
Piru wrote:
@HenryCase

You can't in any sensible way intercept the pointer usage, so the question is irrelevant. You can't "flag" or any other way know when the app is using pointers and/or if it's using pointers correctly or not.


No sensible way intercept pointer usage?


No way, period.  You could make assumptions but at best you would only be right, less than 30% of the time.

Quote

This is linked to the 'can it be done elegantly' question I mentioned previously, whereas this discussion is still at the 'can it be done' stage. Even if the MP solution we find is impractical, but works, then at least we'll know what our real options are.


It can't be done... No system could be made that would be in anyway deterministic.

Quote

So I'll ask again: is it possible to write a program that can scan the binary of another application and know when a pointer has been set?


No. There is nothing special about a pointer... it looks exactly the same as any other kind of number.

Offline bloodline

  • Master Sock Abuser
  • Hero Member
  • *****
  • Join Date: Mar 2002
  • Posts: 12113
    • Show all replies
    • http://www.troubled-mind.com
Re: How to move AROS forward
« Reply #13 on: July 28, 2008, 01:21:46 AM »
Quote

HenryCase wrote:
@bloodline and @Piru
Thank you for your posts, I will now reply to them in one go.

Quote
bloodline wrote:
There is no way to know what the programs are passing to each other.

One thing at a time. First we need to know where the pointers are, then I'll say about my ideas on how we can use this information.


You can't say that!

What if the pointer never actually exists anywhere and is calculated by Task A and then passed to Task B?

Quote

Quote
Piru wrote:
And I answer again: No.

Please explain the reasoning behind your answer, a simple 'No' doesn't help me understand why you don't think there is a way to see pointers in binary code.


I'll answer, again, since I've repeatedly said it before :-)

There is nothing special about a pointer... it is just a number, neither the CPU or the OS has any idea what that number is for.

The CPU just pushes numbers around... buy a good book on 68K ASM and learn the CPU inside out... it shouldn't take more than a few weeks to figure it out... you can't hope to understand this problem until you know a CPU.

Quote

Whilst I wait for your explanation, I would like to ask you another question. How does a CPU know that a program is asking to set up a pointer?


There is no "setting up a pointer", it's just a number to the CPU... nothing special.

Quote

Quote
bloodline wrote:
No way, period. You could make assumptions but at best you would only be right, less than 30% of the time.


ASM is closely linked to the structure of machine code, correct?


ASM is the human readable form of machine code, but often has some high level features to aid coding.

Quote

Also, ASM strives for efficiency.


Not really... My ASM code was far from efficient ;-)

Quote

 I would argue that there are only so many ways to set a pointer in ASM efficiently, and therefore only so many binary structures we would need to scan for to get a higher accuracy than 30% (where you got that figure from I don't know).


My 30% figure was if you tried to assume what the tasks were sending to each other.

It really doesn't matter what a task does in it's own address space. The problems occur when other tasks attempt to access structures which are not in their address space.

Please get over this obsession with pointers... they are not going to solve any problems.

The problems are with the design of AmigaOS... not the CPU or the programming languages used!!

Quote

 I found a couple of ways of setting pointers in x86 ASM (I'm assuming it's x86 ASM anyway) from this page (Google cache version):
cs.colgate.edu/faculty/nevison/cs201web/lectureNotes/arraysandpointers.ppt
========================
clear2:
add  $t0, $a0, $zero    # p = addr of array[0]
forp:   sw   $zero, 0($t0)      # memory[p] = 0
addi $t0, $t0, 4        # p = p + 4
sll  $t1, $a1, 2        # $t1 = 4 * size
add  $t1, $t1, $a0      # $t1 = addr  array[size]
slt  $t2, $t0, $t1      # p < &array[size] ?
bne  $t2, $zero, forp   # if so, continue
endfor:
=========================
Pointer version assembly code
clear2:
add  $t0, $a0, $zero    # p = addr of array[0]
sll  $t1, $a1, 2        # $t1 = 4 * size
add  $t1, $t1, $a0      # $t1 = addr  array[size]
forp:   sw   $zero, 0($t0)      # memory[p] = 0
addi $t0, $t0, 4        # p = p + 4
slt  $t2, $t0, $t1      # p < &array[size] ?
bne  $t2, $zero, forp   # if so, continue
endfor:
========================
Why is it impossible to scan for the binary versions of the ASM code above?


This looks like PPC asm... which I really don't like and was never meant for humans to read... but you miss the point, the CPU doesn't see the high level part of the code. it just adds... subtracts... shifts... ands... ors... nots... numbers! It really doesn't have any idea what the programmer is trying to do, it can't know, and nothing outside of the CPU can know either!

What you are presenting here is totally irrelevant to the problem, which is actually very simple. AmigaOS doesn't know what programs are saying to each other or how programs interact with each other.... AmigaOS allows the programmer to make the programs work in anyway he chooses, that is a design feature of AmigaOS to save memory and be quick on a 7Mhz CPU... Remove that feature and nothing written for AmigaOS works anymore.

Once you add MP, you have to impose restrictions on the programmer and provide him with clearly defined interfaces and methods for achiving a task.

-Edit- I want to add that the pointer problem is deeper than just sharing Data... Code is also shared on AmigaOS, the entire library system is built around the idea than one executable and jump into and run code in another executable... something totally alien to a Memory protected OS...

Offline bloodline

  • Master Sock Abuser
  • Hero Member
  • *****
  • Join Date: Mar 2002
  • Posts: 12113
    • Show all replies
    • http://www.troubled-mind.com
Re: How to move AROS forward
« Reply #14 on: July 28, 2008, 10:47:00 AM »
Quote

SamuraiCrow wrote:
@HenryCase

To scan for those binary equivalents you'd have to know the difference between code and data.  If you write a patcher to go through and change all of the binary instances of memory access outside the main chunk of code, you'd probably end up inadvertently changing some constant data to something different.


I would actually be surprised if you even could identify the memory access, you have no way to know what and where that pointer is going... thus why a JIT can only translate code blocks up to the jumps.

Quote

In short you'd have to go into every program by hand and write patch code for every instance of pointer access.


And even if you could do that, it doesn't solve the problem... AmigaOS programs are designed at a fundamental level to interact intimately, you would have to rewrite the program completely.

Quote

Why not just get the source codes of everything you'd ever want to run on AROS and recompile it for MacOSX?  You'd save yourself a lot of headaches and the rest of us a lot of hassle. :-x


I agree... but you'd have a hell of a time porting a program from AmigaOS to MacOSX... I would rather just rewrite the program.