Amiga.org
Operating System Specific Discussions => Amiga OS => Amiga OS -- Development => Topic started by: kas1e on November 11, 2004, 09:33:40 AM
-
Hi all.
Today I write a little program which read data per one byte to buf1 and must (in loop) adding per one byte to buf2:
--cut---------------------
char buf1[1];
int size=0;
char *buf2=new char[size]; // ????
main()
{
while(recv(sockfd,buf1,sizeof(buf1),0)
{
memcpy(buf2+size,buf1,sizeof(buf1));
size++
};
printf("%s\n",buf2);
};
--cut---------------------
I mean buf2 size is unknown, and must change. On win32 (bcc compiler), this kind of strings:
int size=0;
char *buf2=new char[size];
give me all what i need, but on amiga (sasc):
==================
char *buf2=new char[size];
file.c 37 error 9: Undifined identifier "new"
==================
char *buf2=new char[size];
file.c 37 error 57: semi-colon expected
==================
char *buf2=new char[size];
file.c 37 error 77: identifier expected
So, any help will be very good. Or how i can fix identifier, or maybe any other way for solve this problem. Thnx
-
sounds like you are trying to use a C++ code on a C compiler. The new C++ function is called malloc in C. I think the syntax is:
char *buf2 = (char*) malloc((sizeof(char)*"an integer"));
if I remember corretly
btw:
int size=0;
char *buf2=new char[size];
seems wrong because you make a array with no length, hence nothing will be allocated! size should at least be size=1 the you would get a single char in your array. otherwise you get a memory violation.
memcpy dosen't make allocations only copies between memory-locations, hence it could/should fail if the array is zero bytes long.
I might be wrong here, and new char[0] sees that you want to make a single char as in new char or new char[1]
-
Looks like this would not work even if it would be C++
int size=0;
char *buf2=new char[size]; // ????
this would do nothing since size if 0.
If you want to create an array use a size greater then 0 or end up with zero elements.
Arrays will not grow on their own
memcpy(buf2+size,buf1,sizeof(buf1));
size++
This 'buf2+size' will run out of your allocated space soon and cause all kinds of undetermained behaviour.
If you really want arrays to grow dynamically you should create a new one that is larger than the previous. Copy the previous in the newly created one and then delete the previous to prevent memory leak. I don't know if this will affect memory fragmentation though.
Good luck.
-
sounds like you are trying to use a C++ code on a C compiler. The new C++ function is called malloc in C. I think the syntax is:
char *buf2 = (char*) malloc((sizeof(char)*"an integer"));
if I remember corretly
You don't.
(sizeof(char) * "string") is nonsense, you need to give it the size of the memory to allocate.
For example:
char *str = "a string";
char *buf = malloc(strlen(str) + 1);
if (buf)
{
strcpy(buf, str);
...
free(buf);
}
-
@Piru
HAHA!.. okey think you misunderstood me:
malloc( sizeof(char) * "an integer" ); as in
malloc( sizeof(char) * 5); for exampel :-)
here you would get an array og 5 chars :-)
-
@nex4060
That was rather confusing, then.
-
>If you really want arrays to grow dynamically you should >create a new one that is larger than the previous. Copy the >previous in the newly created one and then delete the >previous to prevent memory leak.
so, i need :
read from socket 1byte by recv, and put to second buffer.
read agayn 1byte, and append to second buffer. and agayn
and agayn,append and append. buffer2 can be or to small or to large - any. so, yes i need "arrays to grow dynamically".
only that i need:
1. buffer 2 with dynamically size. what best way for it ?
on sasc i mean. (example will be very good).
2. append buffer1 to buffer2 in loop. can't buf2+size? i use this on win32, and it work nice.
-
btw:
char buf1[1];
int size=0;
char *buf2=malloc(sizeof(char)*size);
while(recv(new_fd,buf1,sizeof(buf1),0))
{
memcpy(buf2+size,buf1,sizeof(buf1));
size++;
printf("size: %d\n",size);
printf("buf2: %s\n",buf2);
};
compile,run:
size: 1
bif2: a
size: 2
bif2: aa
size: 3
bif2: aaa
size: 4
bif2: aaaa
and next - hardcore reboot/red window of death/etc. why ?:)
on win32 this work good. and i try to change on win32/bcc
this string:
char *buf2 = new char[size];
to
char *buf2=malloc(sizeof(char)*size);
and compiler said to me that can't convert void * to char ..
-
and next - hardcore reboot/red window of death/etc. why ?:)
Because the code overwrite memory that is not allocated.
-
you have made an memory-voilation.
you have to make a new array and copy the old one. memcpy dosn't make a new array or enlarges it for you!
char buf1[1];
int size=1;
char *buf2=malloc(sizeof(char)*size);
char *temp = 0;
while(recv(new_fd,buf1,sizeof(buf1),0))
{
temp = malloc(sizeof(char)*size);
memcpy(temp,buf2,sizeof(buf2));
memcpy(temp+size-1,buf1,sizeof(buf1));
free(buf2);
buf2 = temp;
temp = 0;
size++;
printf("size: %d\n",size);
printf("buf2: %s\n",buf2);
};
that should work
-
memcpy(temp,buf2,sizeof(buf2));
This is wrong. sizeof(buf2) is sizeof(char *), size of a pointer, 4 with 32bit addressing systems, 8 with 64bit addressing ones.
It should be memcpy(temp,buf2,size-1);
Anyway, this code is pretty inefficient, it should use larger buffer size, and preferably linked list of chunks, rather than excessive copying which kills cache.
-
Not at my miggy at the mo but something like...
------------- 8< --------------
// Could do char buf1 here and pass &buf1 to function
char buf1[1];
// Initialise to 0 elements
int size=0;
// Initialise to NULL so realloc works properly!
// Could do "char *buf2 = NULL;", its the same thing!
char buf2[] = NULL;
while (recv(new_fd,buf1,sizeof(buf1),0))
{
// Increase size of array
size++;
// Reallocate buffer (+1 for terminator!)
realloc(buf2,(size+1)*(sizeof(char));
// Copy character (-1 because array starts at 0)
buf2[size-1] = buf1[0];
// Reset terminator
buf2[size] = '\0';
// Print status
printf("size: %d\nbuf2: %s\n",size,buf2);
}
--------------- >8 ------------
should do it.
-
// Could do char buf1 here and pass &buf1 to function
char buf1[1];
// Initialise to 0 elements
int size=0;
// Initialise to NULL so realloc works properly!
// Could do "char *buf2 = NULL;", its the same thing!
char buf2[] = NULL;
while (recv(new_fd,buf1,sizeof(buf1),0))
{
// Increase size of array
size++;
// Reallocate buffer (+1 for terminator!)
realloc(buf2,(size+1)*(sizeof(char));
// Copy character (-1 because array starts at 0)
buf2[size-1] = buf1[0];
// Reset terminator
buf2[size] = '\0';
// Print status
printf("size: %d\nbuf2: %s\n",size,buf2);
}
This code is broken. It crashes horribly when realloc can't resize the allocation, but need to allocate new memory area and move the data. Return value of realloc must be tested for failure or new address.
-
Considered a class that manages a dynamically grown linked-list?? That way you don't have to recopy everything into a brand new array every time you get a new piece of data.
-
I haven't read all of this thread, so I might be missing the obvious:
Why grow an array in tiny increments?
Make a linked list of allocation buffers, each of which contains space for a a few kb of data.
Then simply traverse the list in an outer loop, filling the current buffer and keeping track of how full it is. When it is full, add a new buffer to the list and start from there. You only need to keep track of the size of the current buffer being filled and the overall amount of data recieved.
Disadvantages:
Your data is stored in a noncontiguous way, divided into blocks of several KB. Random access to the data is therefore not efficient when scattered accesses span several blocks.
You can, however traverse the list relatively easily and sequential access would not be a huge performance hit.
Advantages:
No limit to the amount of data that can be recieved (until memory fills, basically). You never copy anything at any point. Your blocksize, inclusive of link pointers can be carefully chosen to give a single block that matches the system page size. This can be an effective optimisation and help reduce framgentation.
If you really need random read access, there is an alternative.
Allocate an array of pointers that point to sequentially allocated blocks of data. These would be all the same size, again several KB each.
You simply fill the first buffer (pointed to by buffers[0]), once that is full, you allocate another, pointed to by buffers[1] etc.
Random access can then be achieved by simply breaking down the supplied index into its block number and offset into the block.
Advantages:
Likely even faster than the linked list for sequential access and dramatically faster for random access.
Disadvantages:
Unless you periodically grow your buffers array, it has an implicit limit to the amount it can hold.
-
Its broken and crashes horribly? A bit harsh matey. I was working from memory (no pun intended).
You are correct though a check on the return value of realloc should be made. If realloc fails then you are out of memory and so are screwed anyway!
Oh, and dont forget to free buf2, or it may break and crash horribly taking the world with it etc..
// Could do char buf1 here and pass &buf1 to function
char buf1[1];
// Initialise to 0 elements
int size=0;
// Initialise to NULL so realloc works properly!
// Could do "char *buf2 = NULL;", its the same thing!
char buf2[] = NULL;
while (recv(new_fd,buf1,sizeof(buf1),0) && (size>0))
{
// Increase size of array
size++;
// Reallocate buffer (+1 for terminator!)
if realloc(buf2,(size+1)*(sizeof(char))>0)
{
// Copy character (-1 because array starts at 0)
buf2[size-1] = buf1[0];
// Reset terminator
buf2[size] = '\0';
// Print status
printf("size: %d\nbuf2: %s\n",size,buf2);
}
else
{
// realloc failed
printf("No more memory!\n");
// force loop exit
size = -1;
}
}
free(buf2);
-
@PhatBoiCollier
The code is still broken and crashes horribly.
If realloc() cannot expand the current memory allocation, it returns different pointer than the one passed in. In that case, you poke already free memory -> boom. If that doesn't nuke, it will crash at double free when calling realloc/free for the wrong ptr.
-
Ok, boys, calm down.
This isn't helping the thread owner.
More advanced memory management is really important thing, but as you can see it is now little bit beyond the current skills.
buffer2 can be or to small or to large
The only important thing is that it is not too small. If it is too large you just simply have some memory allocated that is not used. However from the nature of your code this memory is not used yet, but most probabily will be needed very soon. So every program which tries to be efficient, will allocate more memory than single byte and remember how much is used. When more memory is needed then the proper action is taken (I don't want to describe the possibilities of those actions now).
on win32 this work good. and i try to change on win32/bcc
This works 'by mistake'.
In every compiler, the memory allocating functions internally asks Operating System the number of bytes that simply has to be NOT LESS then requested from the programmer. However the programmer doesn't know that exact value and is not allowed to use more than he requested.
Simply win32 asks internally for bigger buffer than SAS/C. this buffer by coincidence was big enough for your example.
(I know I simplified the problem, Piru.)
But this is not a rule and you never should do this.
The idea of PhatBoiCollier to use realloc() is really very good in your case (I can explain why, but this can be hard to understand for you).
Being you I would do it this way:
char single_char; // same meaning as char signle_char[1];
// Initialise to 0 elements
unsigned allocated=0;
unsigned used=0;
#define BYTES_TO_INCREMENT 256
int OutOfMemory = 0;
char *current_buff = NULL;
char *next_buff;
//prepare the initial buffer
allocated += BYTES_TO_INCREMENT;
current_buff = malloc( allocated );
if( !current_buff )
{
// NO MEMORY!
OutOfMemory = 1;
return;
}
while (recv(new_fd, &single_char, sizeof(single_char), 0) == sizeof(single_char))
{
if( (used + 1) == allocated ) { // we used up all allocated memory (we reserve 1 character for terminating '\0' at the end)
allocated += BYTES_TO_INCREMENT;
next_buff = realloc( current_buff, allocated );
if( next_buffer )
{ // we have new bigger buffer, the old one does not exist any more
current_buff = next_buff;
}
else
{ // there was no memory left, old buffer still exists to allow us to use what we have until now
OutOfMemory = 1;
break;
}
}
current_buff[ used ] = single_char;
++used;
}
current_buff[ used ] = '\0';
printf("allocated: %u\nused: %u\ncontent: %s\n", allocated, used, current_buff );
if( OutOfMemory )
{
puts( "We run out of memory!" );
}
free( current_buff );
Feel free to use it.
Cheers
-
to PiR:
big thnx, your sample is was good. i wrote all what i want.
"The idea of PhatBoiCollier to use realloc() is really very good in your case (I can explain why, but this can be hard to understand for you)."
plz, if you have some time, i try understand.
btw, maybe anyone have link on good docs about work with memory managment in advance ? amiga/sasc orientation will be good.
-
Hi
So I didn't discourage you... damn... ;-)
It is mayby not that hard to explain, but it surely takes time to describe all the details. I'll try to be short and clear:
realloc() solves you two problems:
1. it manages the memory for you (allocates new buffer, frees the old one, whenever needed).
2. if its needed, it performs a copy of the old buffer to the new one.
From the logical point of view you may think that realloc() always allocates new buffer, performs the copy and frees the old buffer.
But do you remember what I've written earlier about internal calls to OS? The allocated buffer is almost always bigger than requested by the programmer, but you don't know how much bigger.
You do not know. But realloc() can find it out. If really allocated buffer is big enough to serve the new value requested by the programmer (as second argument to realloc()), it is simply not needed to allocate, copy and free. So it happens - if only possible, the old buffer is returned as the new one and no more actions are made. What a great speedup!
You must know that memory allocation is quite difficult task for OS - it must check all free buffers and find the best one (and how to define the best one? the first found that is big enough? and how much of it should be returned?). So for that reasons you shouldn't allocate memory too often (not every single byte).
I have also to mention that the code I wrote is quite effective for what is needed in your code. I mean for transfering of some strings you don't need more complicated memory management. Notice that I defined relatively big INCREMENT (256 bytes), so I guess most of cases it would be enough to allocate only once. More complicated memory management would be needed if you had heavy transfers - a lot of data to read and write, and in unpredictable order.
This solution has also one more advantage dedicated for strings - it keeps all the content in single block, so it is always ready to be printed directly by simply using stdio functions without any tricks.
So for that purposes even professional programs would go this way.
If you're interested in more complicated memory management - this is not that much related with the OS at this level. You just have to learn about basic programistinc structures like lists, trees and hash-tables. Then practice, practice and practice to be able to use them intuitively and without mistakes (which are quite easy to be done and very hard to find afterwards in the working code). Then you'll find always the good answer for every programistic question you'll have while programing.
To explain those data structures it is always better to draw the pictures than to explain everything with plain words - it just simply doesn't work too well on the imagination that way.
All the best on your quest. :-)