New improved intuition.library version from the Kickstart 3.1

[wawrzon]

No. Worse. Actually, most of the stuff should be *removed* from the ROM.

im not exactly in accordance with this statement. yes, a lot of stuff might be moved between rom and disk, but dont forget amiga kickstart plays the role of bios initializing devices available at boot time and sicnce the setups are more various as the genuine it will be the question of flexibility and convenience to have drivers to access at least usb if not sata directly in rom. this as example, since i did that.

The only way to *solve* the problem is to make a clean-room implementation of the AmgiaOs code in first place. Yes, AROS does that. Thus, why not contribute there? Clean room implementation means: Person A will reverse engineer, will provide knowledge (but no code) to person B, who reimplements the interface as found from person A. To my very knowledge, this is a valid and legal approach to my very knowledge (but, IANAL). Actually, intuition interfaces are fully documented, thus not such a big deal. And if you want to ask people about internals, hey, you know where to look for.

exactly.

One way or another, I think it is a completely superfluous undertaking, only fragmenting AmigaOs further, without actually offering the chance to resolve any single problem the platform has.

not in my eyes, but if so, then its not the community fault. chances were there, for both sides, the enterprises and the consumers. if there was communication it could even actually take off for both. there wasnt. the community takes its path. sorry for those who missed the opportunity.

[kolla]

Actually, most of the stuff should be *removed* from the ROM. Or are you suggesting that every time somebody makes an update users should burn a new ROM? RAM is cheap enough these days, even for Amiga. Loading the code into RAM costs nothing, it loads fast enough, and it is extremely simple to update it there.

No it isn't "extremely simple". How many times haven't we seen people mess up their setups because suddenly their Amiga booted with old scsi.device and old filesystem? Yes, many of us _do_ build our own updated kickstarts, that we softkick, there are flashable kickstart ROM replacements, we use custom kickstart files for UAE, we use custom kickstart files on FPGA systems. And yes, we do remove things from kickstart too, but certain things just need to go in for the system to be able to boot.

OS3.9 kickstart on my Mimimig, the system is 68000 and there is only 2MB chipram and 1.5MB "slow" RAM available, it's much more efficient to build custom kickstart than to mess around with loadmodule.

[Heiroglyph]

Or get new bugs because the new, ehem, author doesn't understand why particular choices were made, or at least, cannot read the sources...  

I was referring to legitimate bugs. There's always the possibility that some software was written not against the API as documented, but rather to the observed behavior.

It may not be an issue, but that's the only reason I can see for breaking existing applications.

No. Worse. Actually, most of the stuff should be *removed* from the ROM. Or are you suggesting that every time somebody makes an update users should burn a new ROM? RAM is cheap enough these days, even for Amiga. Loading the code into RAM costs nothing, it loads fast enough, and it is extremely simple to update it there. You can even write-protect the code in RAM if you want to. (MuProtectModules) So, there is absolutely no advantage of placing code in ROM.  

I go back and forth on this issue but I tend to agree with you.

You can either try to put more in ROM or provide a more complete disk loading mechanism. (probably a few other options I'm not thinking of)

For a while he's been trying to get a few more items available early in the boot process, such as working RTG, and packing more into the ROM is one valid way of getting there.

With the lack of development in general, having the libraries in a ROM starts to look appealing too.

In the end, it's no improvement - the reverse engineered code does not loose its copyright just because it was reverse engineered, so it cannot be published, on a legal basis. Thus, whether anyone publishes the original source, or Cosmos puts his sources hack onto a public server is no difference. The difference is only *who* takes the risk, but not *why* or *if*. The problem remains the same problem, either way.

It's still risky, but I think fewer and fewer people care. Leaving it alone isn't getting us anywhere either because the owners don't seem to care about the code until they can litigate over it.

The only way to *solve* the problem is to make a clean-room implementation of the AmgiaOs code in first place. Yes, AROS does that. Thus, why not contribute there?

I can't speak for why he doesn't work with AROS.

I don't contribute because I don't like the direction they took. It seems like they work backwards from x86 while adding third-party features, making it a massive, untestable, never ending ordeal.

IMHO, they should have replaced one library from the ROM at a time on 68k before they even started on disk libraries.

That ROM replacement would have given us a start on owning the OS as a community and the ability to legally make the changes you mentioned such as loading from disk.

One way or another, I think it is a completely superfluous undertaking, only fragmenting AmigaOs further, without actually offering the chance to resolve any single problem the platform has.

He's not saving the platform, but he's having fun and quite a few people are interested in it. No more of a loss than playing a game or reading a book, yet a few people will enjoy using his changes.

[psxphill]

Ok, so let's think this to the very end, will we? If I call a device, I need to know the memory type it is able to handle and allocate that (is there a MEMF flag for every memory type a device needs?

Generally there is.

As for example, Zorro-II memory on *this* GVP board, or 32-bit memory on the CPU-board?). What if the caller needs the data in other memory regions? For example because it is an audio sample? Or a graphic for the graphics board? Or a program to load into 32-bit board memory? Thus, a caller has to create an off-target buffer, load the data in there and then copy it around.

I'm not aware that this is a general problem. i.e. a SCSI controller only has ram on it as a convenience to the user for reducing the number of cards required. If there is an additional card that has ram with the same MEMF flag then that is also acceptable (usually 24 bit dma).

I agree that there are some theoretical issues with MEMF and it is also inconvenient as it requires external knowledge and it would have been better if amiga had been forced into providing a better solution. My argument is that having a central system which coordinated memory allocation between components so they are as optimal as possible and only if that all falls down a fallback that allocates buffers is a much better solution than what you propose.

Or to put it in a different way: There's a reason why every other operating system I know handles it the way I consider the correct way (off-side buffers for devices that cannot DMA into all regions), only AmigaOs doesn't.

AmigaOS is the only one that sends messages without copying them too. I understand your argument, I understand why the other operating systems do it that way. I don't buy that it's the correct way that AmigaOS should work.

The FFS Mask/BufMemType flags are workarounds CBM invented because the trackdisk.device was broken and couldn't work with FFS out of the box. That's really all. Then it became an excuse for lazy device implementations, unfortunately.

No. The Mask for was a supposed broken hard disk. BufMemType was for trackdisk to avoid copies, not because it was broken. devices not implementing side buffers was on purpose, not because of laziness. AFAIK trackdisk allowing fast ram transfers had more to do with mfm decoding being faster when using the cpus available at the time than with the blitter (due to the number of passes involved with the blitter).

I've been in a similar position to you where I've justified each component doing copies on a framework that I built, but it all ended up being memory hungry and slow. On a further redesign I was able to reverse my decision and the difference was visible.

[kolla]

I don't contribute because I don't like the direction they took. It seems like they work backwards from x86 while adding third-party features, making it a massive, untestable, never ending ordeal.

IMHO, they should have replaced one library from the ROM at a time on 68k before they even started on disk libraries.

Main purpose of AROS/m68k kickstart really was just to have enough for launching games and demos with WinUAE, and for that it has come a long way.

[wawrzon]

Main purpose of AROS/m68k kickstart really was just to have enough for launching games and demos with WinUAE, and for that it has come a long way.

that, and also aros has its own dynamics of development that happen mostly on x86. no wonder, 68k is maintained since not such a long time. it has its advantages, that features of the major platforms become eventually available on 68k, and it has disadvantages since 68k doesnt enjoy so much focus. but generally one gets along with the other, the aros modules improve on the genuine fuctionality while as i observe the developers care very much for backwards compatibility and it has greatly improved since 68k branch become active.

there is still a number of unimplemented features and there is a number of lacking speed optimizations, such as c2p conversions that must take place on planar displays because aros internally handles gfx as chunky afair.

this all will hardly improve if none gets engaged and everybody waits with hands in their pockets. the majority of aros devs is inerested to implement amiga system on fast up to date hardware, which is a reasonable goal, wouldnt try to convince them otherwise. for 68k it was a tradeoff from the start with. so if you want to have regular progress on amiga 68k compatible patform you must contribute to lobby of aros 68k active testers, users and devs. thats it.

[guest11527]

I'm not aware that this is a general problem. i.e. a SCSI controller only has ram on it as a convenience to the user for reducing the number of cards required. If there is an additional card that has ram with the same MEMF flag then that is also acceptable (usually 24 bit dma).

Unfortunately, that is not true in this generality. For example, there is a batch of A4000 where CBM in their wisdom installed the wrong bus drivers for motherboard RAM (inverting instead of non-inverting). For the CPU, it doesn't make a difference since the data is read and written just "upside down". For DMA, it means that you get all your data inverted from the disk into the RAM. Thus, the only safe destination is really RAM on the card. Yes, surely its broken.

I agree that there are some theoretical issues with MEMF and it is also inconvenient as it requires external knowledge and it would have been better if amiga had been forced into providing a better solution. My argument is that having a central system which coordinated memory allocation between components so they are as optimal as possible and only if that all falls down a fallback that allocates buffers is a much better solution than what you propose.

Then again, where do you suppose to make the cut? At File system level? At user program level? If you make it at file system level, there is no freaking difference. Wether the FFS copies the data, or the device copies the data makes no difference. The data needs copying anyhow. Broken hardware causes problems. If the user has to copy data, you're putting a lot of burden and responsibility at the user level, and given the general leeway users handle AmigaOs requirements, this does not explicitly help to improve the robustness of the overall system. Leave alone you're also creating problems at interoperability level - POSIX and the C-standard lib do not know anything about memory types, thus any program requiring any type of interoperability would require a C-library layer which performs the copy. One way or another, you cannot get rid of the copy except in exceptionally good circumstances, and *that* is something you could arrange *even though* the device implements a copy if it has to.  

I don't have a problem with a device signalling "oh, by the way, I would work faster if you would align buffers in this specific way". I do have a problem with devices "constructed" the way we find them today "Oh, if you don't allocate memory properly, I will just overwrite some other random stuff and probably hang or crash, depending on my mood..."  

AmigaOS is the only one that sends messages without copying them too. I understand your argument, I understand why the other operating systems do it that way. I don't buy that it's the correct way that AmigaOS should work.

AmigaOs passes by reference because that's simply the only thing that could reasonably be done back then with the limited resources of the 68K. Other systems had no multithreading at all (Apple, MS) and would require running in circles to create the illusion (MacOs copying entire patch-tables for its system resource to keep programs happy, as in "SetFunction() would be task specific". If you would construct a system nowadays, you would probably just copy the messages from one address space to another, just to make sure that you isolate tasks properly. But that's an entirely different discussion. At least theoretically, message passing by pointers is working. But given that there is not even an interface to query memory requirements of devices makes the current system completely corrupt. The end user simply "has to know" what a device can do, and there is not even a central repository to store such information. It is exclusively used by the filing system, but *other* programs having to access the harddisk somehow have to know the right values "by magic". Or not - and create a disaster.  

I've been in a similar position to you where I've justified each component doing copies on a framework that I built, but it all ended up being memory hungry and slow. On a further redesign I was able to reverse my decision and the difference was visible.

But we're talking about AmigaOs, and as I already said: At *some* point, the copy has to be made. At file system level, or at device level. The question is just *where*. And the answer is quite obvious: Whoever created the mess must clean up. So it's the device because that's the only point where all the knowledge about constraints and restrictions is known. Hopefully. The filing system or the end user can hardly know all the details.

[OlafS3]

I was referring to legitimate bugs. There's always the possibility that some software was written not against the API as documented, but rather to the observed behavior.

It may not be an issue, but that's the only reason I can see for breaking existing applications.



I go back and forth on this issue but I tend to agree with you.

You can either try to put more in ROM or provide a more complete disk loading mechanism. (probably a few other options I'm not thinking of)

For a while he's been trying to get a few more items available early in the boot process, such as working RTG, and packing more into the ROM is one valid way of getting there.

With the lack of development in general, having the libraries in a ROM starts to look appealing too.



It's still risky, but I think fewer and fewer people care. Leaving it alone isn't getting us anywhere either because the owners don't seem to care about the code until they can litigate over it.



I can't speak for why he doesn't work with AROS.

I don't contribute because I don't like the direction they took. It seems like they work backwards from x86 while adding third-party features, making it a massive, untestable, never ending ordeal.

IMHO, they should have replaced one library from the ROM at a time on 68k before they even started on disk libraries.

That ROM replacement would have given us a start on owning the OS as a community and the ability to legally make the changes you mentioned such as loading from disk.



He's not saving the platform, but he's having fun and quite a few people are interested in it. No more of a loss than playing a game or reading a book, yet a few people will enjoy using his changes.

As Wawa said we need people that contribute to Aros 68k. You do not contribute because of the direction it takes? What direction do you mean? The thread is about 68k so we do not talk (or be in interested in) the direction X86 takes. It is very compatible already, it runs most of the newer applications, many games, WHDLoad and many others. You can replace almost everything easily by copying 68k files, I do that extensively on my distribution. I use Magellan as desktop, I use MUI38 instead of Zune, I have added tons of components from 68k world, original are of course basic libs like dos.library, gadtools, intuition, AHI and CybergraphX 3. On UAE it runs very well, what still needs more love is supporting classic hardware (expecially optimizations that it runs faster). I do not more know what I can do additionally to persuade people what big chances it offers. I think of making a survey and perhaps a bounty for that, then people will have a chance to show interest, if not I personal will concentrate on Aros 68k on emulation.

[OlafS3]

Main purpose of AROS/m68k kickstart really was just to have enough for launching games and demos with WinUAE, and for that it has come a long way.

Wawa explained it very good. People (both users and developers) on 68k have to decide what they want, hack around with the old codebase, adding patches that do not really make it better or supporting a new clean platform that makes it easy to integrate patches in the OS (something that already has happened). Of course progress does not happen if people only sit there twiddling thumbs and moan about the situation. If people want future the choice is clear.

[Cosmos Amiga]

Wawa explained it very good. People (both users and developers) on 68k have to decide what they want, hack around with the old codebase, adding patches that do not really make it better or supporting a new clean platform that makes it easy to integrate patches in the OS (something that already has happened). Of course progress does not happen if people only sit there twiddling thumbs and moan about the situation. If people want future the choice is clear.

No : we need only one direction, and all together... And the RIGHT direction (success), not the many dead-ends...

But it's impossible for many reasons (money, old fight PUP/WOS, big melon...)

[guest11527]

No : we need only one direction, and all together... And the RIGHT direction (success), not the many dead-ends...

Yeah, very funny. Do you decide what the right direction is? Or to put it into a different perspective: Consider it becomes open source: Then there is no way to "prevent a wrong direction" because everybody can branch.

[psxphill]

But we're talking about AmigaOs, and as I already said: At *some* point, the copy has to be made. At file system level, or at device level. The question is just *where*. And the answer is quite obvious: Whoever created the mess must clean up.

Your solution is not obvious if you want the best speed. Forcing everything in the system to take copies is the worst solution for performance and for the sanity of the device developers.

A good solution would allow you to stream data from a file on a hard disk into memory on a sound card without any cpu copying involved ever with the hard disk device and the sound card device negotiating how to do it through an exec api. If copying was required then it would be exec that would do it.

Then again, where do you suppose to make the cut? At File system level? At user program level? If you make it at file system level, there is no freaking difference. Wether the FFS copies the data, or the device copies the data makes no difference.

Why do you need to copy the data? If you're streaming data then buffering it is pointless. If you've told the file system to buffer your file then it makes sense for the file system to take copies, but that should be optional.
 

Unfortunately, that is not true in this generality. For example, there is a batch of A4000 where CBM in their wisdom installed the wrong bus drivers for motherboard RAM (inverting instead of non-inverting). For the CPU, it doesn't make a difference since the data is read and written just "upside down". For DMA, it means that you get all your data inverted from the disk into the RAM. Thus, the only safe destination is really RAM on the card.

I've not heard about this. Any zorro transfer into motherboard ram (chip and fast) is broken? That definitely would require the type of system I'm proposing as it's the motherboard that is broken and making every device avoid dma'ing into motherboard ram is a horrible idea.

[Minuous]

aros modules improve on the genuine fuctionality while as i observe the developers care very much for backwards compatibility

AROS is no solution, it is still missing functionality that was in OS3.5 15 years ago...

[Mazze]

AROS is no solution, it is still missing functionality that was in OS3.5 15 years ago...

Luckily it's Open Source. That gives everyone the possibility to implement missing features.

[guest11527]

Your solution is not obvious if you want the best speed. Forcing everything in the system to take copies is the worst solution for performance and for the sanity of the device developers.

I'm not forcing to make copies. I'm requiring that devices provide the data where the user requested them. If the device cannot copy the data into the target (via DMA or otherwise), then it must be copied. But only then. Which option is the best depends on the device.  

A good solution would allow you to stream data from a file on a hard disk into memory on a sound card without any cpu copying involved ever with the hard disk device and the sound card device negotiating how to do it through an exec api. If copying was required then it would be exec that would do it.

How can it? And why should exec do it? And when should it do it?  

Why do you need to copy the data?

Because the target memory might not be reachable by the device. Haven't we had this before? If the user requested data in motherboard mem, but DMA into motherboard mem is broken, then on the way from the device through the FFS to the user *some* component must copy the data (but only then). You can put the burden on the user program (bad, because every program has to reimplement the same logic), the filing system (right now, bad, because the filing system does not know the requirements of the device) or the device itself. Take your poison.

I'm not suggesting that everything must be copied. I'm saying that a device should make sure that, by whatever means, the data ends up where the user requested it. And not "depending on the mood of the device" somewhere completely else, like the trackdisk device did. If the only means is to copy the data, so might it be. Another option would be be to fall back to PIO...  

I've not heard about this. Any zorro transfer into motherboard ram (chip and fast) is broken?

On this specific series, yes. It's documented in the Guru-ROM manuals, if you care. PIO works (CPU pulls from Zorro are ok), DMA does not (if the device initiates the transfer).  

 That definitely would require the type of system I'm proposing as it's the motherboard that is broken and making every device avoid dma'ing into motherboard ram is a horrible idea.

No, not every device, that's the point. If the device sits on the CPU board, it can DMA into the CPU board memory fine. The Guru-ROM takes such decisions, actually. The CPU can do the rest if required. One way or another, if the target buffer requested by the user is somewhere in chipmem, there is no other way of doing it - some system component has to copy the data on such a system. The only question is: Which? And which component has enough information to initiate the ideal method of getting data from A to B. (DMA,PIO,Copy). If you ask me, it's surely not the filing system that can.

[guest11527]

No : we need only one direction, and all together... And the RIGHT direction (success), not the many dead-ends...

But it's impossible for many reasons (money, old fight PUP/WOS, big melon...)

It's really kind of ironic. On one hand, you're requesting that we go into the right direction all together. On the other, you are exactly preventing that by avoiding any type of coordination. I'm not even caring how you want to coordinate - either with the owners, or with the folks that attempt an Open Source rewrite. One way or another, you cannot define the direction yourself, you need other people. You can try to work on the original sources and avoid a fragmentation by talking to Hyperion. Or you can try to create an OpenSource look-a-like where your creativity is not hindered by management decisions, but community decisions.  

Yet, you want to do your "own thing", and don't even see that you are perverting exactly these goals...

Well, anyhow. "You" is not "we all together". Maybe you'll understand later... I can't help it anymore.