Tell me how you will find out what the percentage of time various software will spend in forbid without trying it?
Getting accurate results can be difficult but itcould be profiled. At least how many calls to Forbid() or Disable() there are per minute...
You have always been able to write software for AmigaOS which disturbs high priority tasks. If it turns out that this is a problem that needs solving then you could try changing it so that cpu's will only ever be running a task that has the same priority. As high priority tasks are supposed to run for a short period of time then wasting the other cores during that time may not be a big deal. If you have a high priority task on AmigaOS that takes a long time then it becomes unusable (standard priority tasks like workbench won't be allowed to run at all). The priority in AmigaOS is quite fine grained (-128 to 127 IIRC) which means you could also derail this using as many as possible. But there is no reason why software that can take advantage of SMP shouldn't have limitations (like all Tasks that run at the same time have to run at the same priority).
I know what you mean. There is a hope that at least sometimes some software could run on parallel. Even if you get only +10% instead of +100% it is better than nothing. In the future bottlenecks could be removed one by one.
I understand about the Forbid() overhead, if something spins in a Forbid()/Permit() call then it could cause problems. But is that something that any software should need/want to do? We pretty much have the source to all AROS software at this point & this isn't going to affect AROS 68k.
Often Forbid() or Disable() is called indirectly. Take this example:
sillypseudocode()
{
SetTaskPri(SysBase->ThisTask, -128);
PutMsg(port, msg);
while (true)
PutMsg(GetMsg(port));
}
PutMsg() and GetMsg() have hidden Disable() but that is fine because on single core system high priority tasks get scheduled as soon as Enable() is called.
On multicore this would steal almost all available CPU time from each core.
Having Disable() free messaging system would solve this problem but this would have implications to all software. Another solution could be limiting scheduler to not schedule lower priority tasks on other cores as you mentioned.
Anyway, I just wanted to point out that biggest culprit is the OS itself and write some silly example