memory fragmentation

[NianHawk]

would frequent allocation and release of small buffers definitely cause
memory leaks? My application have to allocate small buffer for sending data
and then release it after it is done very frequently. I am not sure whether
it would cause memory fragmentation and thus poor performance? Your help
would be appreciated.

[David Lowndes]

Post by NianHawk
would frequent allocation and release of small buffers definitely cause
memory leaks?

No. If you're releasing everything you've allocated, there is no leak.

Post by NianHawk
My application have to allocate small buffer for sending data
and then release it after it is done very frequently. I am not sure whether
it would cause memory fragmentation and thus poor performance?

It may cause fragmentation, but it depends on the memory manager
you're using and the pattern of usage.

Are you seeing any performance issues that you think are due to
fragmentation, and are you *sure* they're due to fragmentation?

Dave

[Barry Schwarz]

On Fri, 19 Dec 2008 23:20:01 -0800, NianHawk

Post by NianHawk
would frequent allocation and release of small buffers definitely cause
memory leaks? My application have to allocate small buffer for sending data
and then release it after it is done very frequently. I am not sure whether
it would cause memory fragmentation and thus poor performance? Your help
would be appreciated.

Why or how do you think memory fragmentation would cause poor
performance?

[Ben Voigt [C++ MVP]]

Post by Barry Schwarz
On Fri, 19 Dec 2008 23:20:01 -0800, NianHawk

Post by NianHawk
would frequent allocation and release of small buffers definitely cause
memory leaks? My application have to allocate small buffer for sending data
and then release it after it is done very frequently. I am not sure whether
it would cause memory fragmentation and thus poor performance? Your help
would be appreciated.

Why or how do you think memory fragmentation would cause poor
performance?

Because there are more blocks for the heap manager to keep track of? I
doubt the heap algorithms are O(N), but I also doubt they are O(1).

In addition, heap allocation and deallocation are fairly slow operations
which additionally require obtaining a global lock. So it's best to reuse
buffers internally instead of freeing and reallocating, if possible.

Post by Barry Schwarz
--
Remove del for email

[Barry Schwarz]

On Sat, 20 Dec 2008 08:31:59 -0600, "Ben Voigt [C++ MVP]"

Post by Ben Voigt [C++ MVP]

Post by Barry Schwarz
On Fri, 19 Dec 2008 23:20:01 -0800, NianHawk

Post by NianHawk
would frequent allocation and release of small buffers definitely cause
memory leaks? My application have to allocate small buffer for sending data
and then release it after it is done very frequently. I am not sure whether
it would cause memory fragmentation and thus poor performance? Your help
would be appreciated.

Why or how do you think memory fragmentation would cause poor
performance?

Because there are more blocks for the heap manager to keep track of? I
doubt the heap algorithms are O(N), but I also doubt they are O(1).

I expect this is be true on many (even most) systems. But I can also
envision a management algorithm where free-d memory is not reusable
and the manager only keeps track of the address and size of memory
that has not been allocated.

Post by Ben Voigt [C++ MVP]
In addition, heap allocation and deallocation are fairly slow operations
which additionally require obtaining a global lock. So it's best to reuse
buffers internally instead of freeing and reallocating, if possible.

No argument from me. Frequent unnecessary calls to any function will
probably adversely affect performance if for no other reason than the
call and return overhead. But this is an algorithm issue and the
question was about fragmentation affecting performance.

[Giovanni Dicanio]

Post by NianHawk
would frequent allocation and release of small buffers definitely cause
memory leaks?

No, if you properly free every allocated buffer.

Post by NianHawk
My application have to allocate small buffer for sending data
and then release it after it is done very frequently. I am not sure whether
it would cause memory fragmentation and thus poor performance? Your help
would be appreciated.

If you have a need to do frequent allocations of small chunks (e.g. when
you are handling node-based structures like tress, e.g. when processing
XML nodes, etc.), you may want to use a custom allocator, and not the
default VC++'s one.

You may find the following links to be useful:

http://g.oswego.edu/dl/html/malloc.html
http://www.hoard.org/

If you want to integrate your custom allocator with STL, you may want to
read this blog post by Stephan (of VC++ Team):

http://blogs.msdn.com/vcblog/archive/2008/08/28/the-mallocator.aspx

HTH,
Giovanni

[Jim Balson]

Post by Giovanni Dicanio

Post by NianHawk
would frequent allocation and release of small buffers definitely
cause memory leaks?

No, if you properly free every allocated buffer.

Post by NianHawk
My application have to allocate small buffer for sending data and then
release it after it is done very frequently. I am not sure whether it
would cause memory fragmentation and thus poor performance? Your help
would be appreciated.

If you have a need to do frequent allocations of small chunks (e.g. when
you are handling node-based structures like tress, e.g. when processing
XML nodes, etc.), you may want to use a custom allocator, and not the
default VC++'s one.
http://g.oswego.edu/dl/html/malloc.html
http://www.hoard.org/

Haven't looked at one from oswego.edu, but hoard is slow. It is barely
faster than the algorithms shipped by Microsoft. In most cases it's a
lot slower.

Look at ESA:

http://www.cherrystonesoftware.com

Very fast. Highly scalable.

HTH.


Jim