Welcome, Guest. Please login or register.

Author Topic: Extensible file format spec ...  (Read 2916 times)

Description:

0 Members and 3 Guests are viewing this topic.

Offline JoseTopic starter

  • Hero Member
  • *****
  • Join Date: Feb 2002
  • Posts: 2871
    • Show all replies
Extensible file format spec ...
« on: September 04, 2006, 08:21:17 PM »
Hi. I was messing around with a simple variable data size file format I made, wich basically consists of a bunch of offsets to the data and it's size in the beggining of the file. Problem is that if the file gets VERY big it's just not practical to rewrite it all over again each time an update is needed. I understand that the reason for this being cummon practice is that today's storage systems are very fast, so no harm in just rewritting the file again.
Heck, for the use I'll give it it's probably not even worth it but just for the fun here's what I've come up with, if anyone sees something that could be better just post.. :-) Data can be anything. I guess to be a cool thing the only things missing are file data identification and nesting ? IFF comes to mind, but from what I remotely remember it doesn't support file data extension does it ?

(NOte: I know 1st is not an acceptable way to start a C var, it's just a scratch..)

LONG FileSize; /* Could be a quadword :)*/
/* Header 1st Part */
LONG 1stDataElement1stPartOffst;
LONG 1stDataElement1stPartSize;
LONG 1stDataElement2ndPartOffstOffst; /* Offset to 1stDataElement2ndPartOffst. Could be null to terminate 1st Data element. IMPORTANT: 2nd Offset is right after 2nd data */
LONG 2ndDataElement1stPartOffst;
LONG 2ndDataElement1stPartSize;
LONG PadNULL; /* NULL 2 terminate 2nd Data Element */
LONG 1;/*1 Indicates Next LONG is offset to header 2nd part */
LONG Header2ndPartOffst;
DATA
1stData Element1st Part:
...
...
2nd Data Element 1st (and only) Part:
...
...
...
..
LONG 1stDataElement2ndPartOffst;
LONG 1stDataElement2ndPartSize;
LONG PadNULL; /* NULL 2 terminate 1st Data Element IMPORTANT: could be an offset to 3rd part */
1stData Element 2nd Part:
...
...
Header2ndPart
...
LONG 2 /* 2 Indicates Header termination */
\\"We made Amiga, they {bleep}ed it up\\"
 

Offline JoseTopic starter

  • Hero Member
  • *****
  • Join Date: Feb 2002
  • Posts: 2871
    • Show all replies
Re: Extensible file format spec ...
« Reply #1 on: September 04, 2006, 09:27:10 PM »
What ?! Noone ? At least tell me what you think, even if you think it's crap  :-D.
\\"We made Amiga, they {bleep}ed it up\\"
 

Offline JoseTopic starter

  • Hero Member
  • *****
  • Join Date: Feb 2002
  • Posts: 2871
    • Show all replies
Re: Extensible file format spec ...
« Reply #2 on: September 06, 2006, 08:02:20 PM »
@Karlos

Hey! Thnx for the offer but I never programmed in C++ and I don't fell like learning now even if it's easy:) I think I'll make my own just for the fun:)

One interesting thing I noticed is that with extensible data the file ends up having the same problems filesystems face when some data in the middle is deleted: fragmentation. But IIRC there are some functions to tell the filesystem that a part of a file is to be scrapped and it will take care of fragmentation issues itself (if it does at all). How did you handle this issue ?
\\"We made Amiga, they {bleep}ed it up\\"