Creating a new file format (need suggestions)

[Jose]

Hi. I need a good way to save a dynamic data structure that I created to HD. It consists of an Amiga linked list whose elements can be another forward only linked list I created, or a single element (same as those in the forward linked list). The forward linked list elements can also be forward linked lists and so on 8-)
Since I've never done this nor seen any example, can someone give some good advice ?
I'm thinking about creating a file format whose elements have an identifier that identifies the list and element. Nested lists would be saved inside it's lists.... whatever ...I still don't know if there's another way of doing it or any details I shouldn't miss..

[TheMagicM]

10 print "I dont know"
20 goto 10

[Daedalus]

Hmmm... Would it not make sense then to have something recursive like a filesystem-type-thing in the file format? A header which points at offset "index" blocks, and each of these index blocks can contain offsets to data or further index blocks. Each index block having an identifier field which would correspond to one of the lists you were trying to save. It's not something I've ever done myself, but if I was to try, that's the approach I'd take...

[Joshua]

I'd suggest an XML file as you can create as many depths of children as you need.

Read your RKRM's and create your own IFF type. :-D

[mgerics]

Do you mean you want the data saved to HD? I'd think you'd just traverse the list writing out each node's data to a standard format file; read it back into the list when needed (that's what I've done in the past, when I was a ral programmer using C). Or do you want to be able to access any node in the list, and said list can be too big to fit into physical memory? That sounds like a relational (or other) database type of thingy.

[Jose]

@Daedalus

Yes!! Makes much more sense than trying to save all the elements with just an identifier. Even though I think here the search could skip lists by puting the size of each list along with it's identifier. But I'd still have to go through the elements of the first list successively, and then when wanted element is found search through it's elements again if it's another list.
 I guess the difference would be that all the elements (including other indexes) of a list would be referenced in the beginning on it's index. Is that how indexes are normally organized or they can be all condensed in the beggining for faster access ..?

[Jose]

@Joshua
I'm not a programmer, I've never learned XML and I don't want to have the hassle of having to learn it. But maybe it's easy.

@mdma
:-D Yes, sounds like a good idea, I've actually read the IFF specification once, but never used it and forgot it by now... It's a lot more work though, I think.

@mjerics

"Or do you want to be able to access any node in the list, and said list can be too big to fit into physical memory? "

It will most likely allways fit on physicall memory but I want to be able to know in which list each node belongs too after saving to HD.    

[Daedalus]

@Jose

Well, indexes are either referenced from the top of the file, or from the parent index, so the speed issue won't really come into play until you've got hundreds of layers, as your indexes can use offsets to the child indexes or to the data, so a list, say, 4 layers deep, will involve only 4 jumps to definite locations. I suppose the indexes could be put at the start of the file for speed provided they are worked out in memory beforehand. If they can't be calculated before saving they could in theory be put at the end of the file, but then you have the problem that all the data is useless if the file is truncated...

Anyway, still just suggestions, and as you develop I'm sure changes can be made to better suit the situation.