Can auto_ptr<> be used with void and malloc()/free()

[bob]

Hi,

I need to allocate a variable size struct using malloc() and then free it
using free(). Is there a way I could wrap a pointer returned from malloc()
in auto_ptr<>? I've used auto_ptr<> with objects of struct/class but only
when they were allocated using new() operator. AFAIK, it is generally not
safe to allocate with malloc() and then free with delete().

Example:

struct {
int x;
int y;
unsigned char data[1];
} structA;

structA* pa = (structA*) malloc(100);

I'd like to wrap pa pointer in auto_ptr<> so I do not have to worry about
freeing it when I leave a given block.


I think I could do the following:

unsigned char* p = new unsigned char[100];
structA* pa = (structA*) p;
_auto_ptr<unsigned char> ptrA(p);
[...]


Any thoughts?

Thanks.

[John H.]

Post by bob
I need to allocate a variable size struct using malloc() and then free it
using free().

Why? That seems atypical.

Post by bob
Is there a way I could wrap a pointer returned from malloc()
in auto_ptr<>?

Perhaps. See below.

Post by bob
I've used auto_ptr<> with objects of struct/class but only
when they were allocated using new() operator.

This is good. auto_ptr was designed to work with new and delete
(scalar versions).

Post by bob
AFAIK, it is generally not
safe to allocate with malloc() and then free with delete().

That is correct.

Post by bob
struct {
int x;
int y;
unsigned char data[1];
} structA;
structA* pa = (structA*) malloc(100);

Whenever you fine yourself reaching for malloc in C++, you should
question whether that is really what you should be doing. It looks
like you are trying to treat your structure as something that has an
x, y, and then some data which is explicitly one char big, but really
might be more (perhaps 92 bytes in your case). Rather than doing
obscure things with memory, I think you are better of using a
different data structure. Consider making data an std::vector. That
way it can have a dynamic size, and the cleanup is done for you. You
don't have to worry about auto_ptr.

Post by bob
I'd like to wrap pa pointer in auto_ptr<> so I do not have to worry about
freeing it when I leave a given block.

Well in general that can be a fine goal, but in your situation I am
not sure.

Post by bob
unsigned char* p = new unsigned char[100];
structA* pa = (structA*) p;
_auto_ptr<unsigned char> ptrA(p);
[...]

So this is a different problem than the first one mentioned, where
you needed malloc and free? I guess you are trying to accomplish the
same thing with the data structure, but using new/delete instead.
Perhaps it is slightly better than the malloc/free version, but it
still seems to be missing the mark.

Post by bob
Any thoughts?

I think you want to rethink things and I would need to know more about
what you are trying to do to give you good advice.
But if you really want to use auto_ptr with malloc/free you might
consider something like the following. I am not sure if it works
correctly or safely.

#include <memory>
#include <cstdlib>

struct structA
{
int x;
int y;
unsigned char data[1];
void operator delete(void * this_ptr);
private:
void * operator new(unsigned int);
};

void structA::operator delete(void * this_ptr)
{
std::cout << "delete" << std::endl;
free(this_ptr);
}

int main()
{
void * memory = malloc(100);
structA * fake_structA = static_cast<structA *>(memory);
std::auto_ptr<structA> autoptr(fake_structA);
return 0;
}

[legalize+ (Richard)]

[Please do not mail me a copy of your followup]

Post by bob
I need to allocate a variable size struct using malloc() and then free it
using free(). [...]

There are better ways to do this in C++.

The only time you need to use malloc/free is when you are
interoperating with a C (not C++) library.

Personally, when I need to talk to C code, I hide it behind an
abstraction that handles all the interoperability concerns for me.

Part of that is using c'tors and d'tors that handle malloc/free business.

[John H.]

Post by bob
unsigned char* p = new unsigned char[100];
structA* pa = (structA*) p;
_auto_ptr<unsigned char> ptrA(p);
[...]

That might not work very well, as you allocated with new[] and
auto_ptr will deallocate with scalar delete.

[Ulrich Eckhardt]

Post by bob
I need to allocate a variable size struct using malloc() and then free it
using free(). Is there a way I could wrap a pointer returned from
malloc() in auto_ptr<>?

No, auto_ptr uses delete, which you must not use with a pointer returned
from malloc. What I'm not sure about is whether you could override
operators new/delete for your struct and redirect these accordingly to do
The Right Thing(tm). I'd try asking this in comp.lang.c++.moderated.

Post by bob
structA* pa = (structA*) malloc(100);

Use static_cast.

If everything fails, you can also write an auto_ptr-like class yourself. Or
take this one:

http://lists.boost.org/Archives/boost/2007/11/130127.php

Cheers!

Uli

[legalize+ (Richard)]

[Please do not mail me a copy of your followup]

Post by Ulrich Eckhardt

Post by bob
I need to allocate a variable size struct using malloc() and then free it
using free(). Is there a way I could wrap a pointer returned from
malloc() in auto_ptr<>?

No, auto_ptr uses delete, which you must not use with a pointer returned
from malloc. What I'm not sure about is whether you could override
operators new/delete for your struct and redirect these accordingly to do
The Right Thing(tm). I'd try asking this in comp.lang.c++.moderated.

Overriding operator new/delete for a struct is overkill. Just have
the c'tor and d'tor handle the resource acquisition and release just
like any other RAII helper.

[Ulrich Eckhardt]

Post by legalize+ (Richard)

Post by Ulrich Eckhardt

Post by bob
I need to allocate a variable size struct using malloc() and then
free it using free(). Is there a way I could wrap a pointer
returned from malloc() in auto_ptr<>?

No, auto_ptr uses delete, which you must not use with a pointer returned
from malloc. What I'm not sure about is whether you could override
operators new/delete for your struct and redirect these accordingly to do
The Right Thing(tm). I'd try asking this in comp.lang.c++.moderated.

Overriding operator new/delete for a struct is overkill. Just have
the c'tor and d'tor handle the resource acquisition and release just
like any other RAII helper.

No, read the original posting again: The OP wants to allocate a structure
with a _variable_ size. For that, you use an array as last struct member
(in his case one with size 1, but IIRC C99 also allows size zero now) and
allocate extra bytes at the end.

Uli

[legalize+ (Richard)]

[Please do not mail me a copy of your followup]

Post by Ulrich Eckhardt
No, read the original posting again: The OP wants to allocate a structure
with a _variable_ size.

Yeah, big deal. So its variable size. So its allocated by a C
library.

So what?

You can still handle the acquisition and release of this resource with
a c'tor/d'tor pair.

You don't need to override operator new/delete.

Do the simplest thing that could possibly work. Overriding operator
new/delete is more complicated than is necessary.

[Ulrich Eckhardt]

Post by legalize+ (Richard)

Post by Ulrich Eckhardt
No, read the original posting again: The OP wants to allocate a structure
with a _variable_ size.

Yeah, big deal. So its variable size. So its allocated by a C
library.
So what?
You can still handle the acquisition and release of this resource with
a c'tor/d'tor pair.

Yes you can, e.g. with some smart pointer that holds a pointer to such an
object, which is what the OP was asking about. No, not in auto_ptr's,
because that uses delete.

Richard, this structure is not an object as normal objects, where you simply
allocate "sizeof (type)" bytes, invoke the ctor on them and then work with
them. Rather you allocate "sizeof (type) + extra" bytes and then assume
that the array, which is the last field in the struct, has "extra" bytes.

You can not do that in the constructor/destructor, because at that time the
storage for the object was already allocated. In case you suggest using a
vector<byte> for the array in question, I believe you are missing the
point. If the OP decided it's best to use a variable-size array in a
struct, there may actually be a reason for that.

Post by legalize+ (Richard)
You don't need to override operator new/delete.
Do the simplest thing that could possibly work. Overriding operator
new/delete is more complicated than is necessary.

Maybe that _is_ the simplest thing possible.

Uli

[legalize+ (Richard)]

[Please do not mail me a copy of your followup]

Post by Ulrich Eckhardt

Post by legalize+ (Richard)
You don't need to override operator new/delete.
Do the simplest thing that could possibly work. Overriding operator
new/delete is more complicated than is necessary.

Maybe that _is_ the simplest thing possible.

Maybe, but in this case: no, its not.

Even a simple smart pointer wrapper around this C data structure is
simpler for people to understand and maintain than overriding
new/delete.

Overriding new/delete is *hard* to do properly. It should be avoided
unless absolutely required and in this case, it isn't.

[Ulrich Eckhardt]

Post by Ulrich Eckhardt

Post by bob
I need to allocate a variable size struct using malloc() and then free it
using free(). Is there a way I could wrap a pointer returned from
malloc() in auto_ptr<>?

No, auto_ptr uses delete, which you must not use with a pointer returned
from malloc. What I'm not sure about is whether you could override
operators new/delete for your struct and redirect these accordingly to do
The Right Thing(tm). I'd try asking this in comp.lang.c++.moderated.

Without claiming that this is portable, it seems to do the right thing under
GCC 4:

struct var
{
size_t len;
unsigned char content[0];
#ifdef __cplusplus
static auto_ptr<var> create(size_t len)
{
void* ptr = malloc(sizeof (var) + len);
printf("create() = %p\n", ptr);
if(!ptr)
throw std::bad_alloc();
auto_ptr<var> res(static_cast<var*>(ptr));
res->len = len;
return res;
}
auto_ptr<var> clone() const
{
auto_ptr<var> res = create(len);
memcpy(res.get(), this, sizeof (var) + len);
return res;
}
void operator delete(void* ptr)
{
printf("delete(%p)\n", ptr);
free(ptr);
}
private:
// private constructor, use create()
var(){}
// not copyable, use clone()
var(var const&);
// not assignable
var& operator=(var const&);
#endif
};


Cheers!

Uli

[bob]

Thanks to everyone who replied to my post.

It looks like I have to get rid of malloc()/free() or forget about
auto_ptr<> and deal with deallocating memory manually.

For those interested, the reason that I'm using a variable length struct is
that I receive it from a network as a stream of bytes. The first couple of
bytes represent a header and the length of variable size data that follows.
I have a low-level c routine that deals with assembling the packet and then
returns it to clients. Currently the routine allocates ram for the packet
and the client frees it. Quite often logic on the client gets a bit
complicated and ideally I'd like to use auto_ptr<> so I do not need to worry
about freeing the memory allocated in the c routine.

Thanks.

Post by bob
Hi,
I need to allocate a variable size struct using malloc() and then free it
using free(). Is there a way I could wrap a pointer returned from
malloc() in auto_ptr<>? I've used auto_ptr<> with objects of struct/class
but only when they were allocated using new() operator. AFAIK, it is
generally not safe to allocate with malloc() and then free with delete().
struct {
int x;
int y;
unsigned char data[1];
} structA;
structA* pa = (structA*) malloc(100);
I'd like to wrap pa pointer in auto_ptr<> so I do not have to worry about
freeing it when I leave a given block.
unsigned char* p = new unsigned char[100];
structA* pa = (structA*) p;
_auto_ptr<unsigned char> ptrA(p);
[...]
Any thoughts?
Thanks.

[John H.]

Post by bob
Thanks to everyone who replied to my post.
It looks like I have to get rid of malloc()/free() or forget about
auto_ptr<> and deal with deallocating memory manually.
For those interested, the reason that I'm using a variable length struct is
that I receive it from a network as a stream of bytes. The first couple of
bytes represent a header and the length of variable size data that follows.
I have a low-level c routine that deals with assembling the packet and then
returns it to clients. Currently the routine allocates ram for the packet
and the client frees it. Quite often logic on the client gets a bit
complicated and ideally I'd like to use auto_ptr<> so I do not need to worry
about freeing the memory allocated in the c routine.

Is it a requirement that the low-level be C? One thing you might try
is redoing your struct like this:

struct structA {
int x;
int y;
std::vector<unsigned char> data;
};

You can then return this structure by value to the client code. When
they are done using it, the memory will clean itself up. If you don't
like to return by value, you could put the structA into an auto_ptr
and return that:

structA read_socket(socket & sock)
{
unsigned char buffer[256] = {0};
buffer << sock;
structA packet;
packet.data.insert(data.begin(), buffer, buffer+255);
return packet;
}

or

std::auto_ptr<structA> read_socket(socket & sock)
{
unsigned char buffer[256] = {0};
buffer << sock;
std::auto_ptr<structA> packet(new structA);
packet->data.insert(data.begin(), buffer, buffer+255);
return packet;
}

Also as Richard said, you could leverage a constructor/destructor into
the structA to simplify the interface even more.

[bob]

I think that if I introduce any C++ members in the struct then the struct
will no longer be a C struct and therefore I will not be able to cast its
pointer to an array of bytes that I can safely pass to a C routine. At the
moment it is more of a convenience issue and an attempt to avoid any mods to
existing C code.

Post by John H.

Post by bob
Thanks to everyone who replied to my post.
It looks like I have to get rid of malloc()/free() or forget about
auto_ptr<> and deal with deallocating memory manually.
For those interested, the reason that I'm using a variable length struct is
that I receive it from a network as a stream of bytes. The first couple of
bytes represent a header and the length of variable size data that follows.
I have a low-level c routine that deals with assembling the packet and then
returns it to clients. Currently the routine allocates ram for the packet
and the client frees it. Quite often logic on the client gets a bit
complicated and ideally I'd like to use auto_ptr<> so I do not need to worry
about freeing the memory allocated in the c routine.

Is it a requirement that the low-level be C? One thing you might try
struct structA {
int x;
int y;
std::vector<unsigned char> data;
};
You can then return this structure by value to the client code. When
they are done using it, the memory will clean itself up. If you don't
like to return by value, you could put the structA into an auto_ptr
structA read_socket(socket & sock)
{
unsigned char buffer[256] = {0};
buffer << sock;
structA packet;
packet.data.insert(data.begin(), buffer, buffer+255);
return packet;
}
or
std::auto_ptr<structA> read_socket(socket & sock)
{
unsigned char buffer[256] = {0};
buffer << sock;
std::auto_ptr<structA> packet(new structA);
packet->data.insert(data.begin(), buffer, buffer+255);
return packet;
}
Also as Richard said, you could leverage a constructor/destructor into
the structA to simplify the interface even more.

[Tim Roberts]

Post by bob
I need to allocate a variable size struct using malloc() and then free it
using free(). Is there a way I could wrap a pointer returned from malloc()
in auto_ptr<>? I've used auto_ptr<> with objects of struct/class but only
when they were allocated using new() operator. AFAIK, it is generally not
safe to allocate with malloc() and then free with delete().

Although it's certainly good to bear this in mind as a general rule, in the
Visual C++ library, "operator new" actually calls "malloc".

[legalize+ (Richard)]

[Please do not mail me a copy of your followup]

Post by Tim Roberts
Although it's certainly good to bear this in mind as a general rule, in the
Visual C++ library, "operator new" actually calls "malloc".

There is nothing in the C++ standard that says this must be the case,
however. Its an implementation detail. You shouldn't depend on it.

Also, array new/delete does more than just call malloc/free.

It is an error to call free on memory allocated with new.

It is an error to call delete on memory allocated with malloc.

Don't write code that "works by accident".
Write code that "works by design".

[Sousuke]

Post by bob
Hi,
I need to allocate a variable size struct using malloc() and then free it
using free().  Is there a way I could wrap a pointer returned from malloc()
in auto_ptr<>?  I've used auto_ptr<> with objects of struct/class but only
when they were allocated using new() operator.  AFAIK, it is generally not
safe to allocate with malloc() and then free with delete().
struct {
  int x;
  int y;
  unsigned char data[1];
} structA;
structA* pa = (structA*) malloc(100);
I'd like to wrap pa pointer in auto_ptr<> so I do not have to worry about
freeing it when I leave a given block.
unsigned char* p = new unsigned char[100];
structA* pa = (structA*) p;
_auto_ptr<unsigned char> ptrA(p);
[...]
Any thoughts?

Sorry to drop in late, but here's something no one suggested: use
shared_ptr, which supports custom deallocation. (auto_ptr implements
single-owner ownership, whereas shared_ptr implements shared
ownership, so I think shared_ptr can be used as a replacement for
auto_ptr if you want.)

#include <stdlib.h>
#include <memory>

using namespace std::tr1;

// Your C-compatible struct
struct PacketData
{
int x;
int y;
unsigned char data[1];
};

struct PacketDataDeleter
{
void operator()(PacketData* pd)
{
free(pd);
}
};

// Suppose this is the function that reads from the
// network and assembles a PacketData
PacketData* ReadPacketData();

// Example usage
void DoSomething()
{
shared_ptr<PacketData> pd(ReadPacketData(), PacketDataDeleter());

// use pd
}

[Ulrich Eckhardt]

Post by Sousuke
struct PacketDataDeleter
{
void operator()(PacketData* pd)
{
free(pd);
}
};

[...]

Post by Sousuke
shared_ptr<PacketData> pd(ReadPacketData(), PacketDataDeleter());

Can't you pass "&free" as deleter to the constructor?

Anyway, replacing exclusive ownership with shared ownership can work, but
doesn't have to. What I often have is exclusive ownership of mutable data
and shared ownership of constant data. This works even in the context of
multiple threads and actually is a big help there. Having shared mutable
data would mandate having a mutex for it, which complicates things.

Uli

[Sousuke]

Post by Ulrich Eckhardt

Post by Sousuke
struct PacketDataDeleter
{
    void operator()(PacketData* pd)
    {
        free(pd);
    }
};

[...]

Post by Sousuke
    shared_ptr<PacketData> pd(ReadPacketData(), PacketDataDeleter());

Can't you pass "&free" as deleter to the constructor?

Yes... The Boost documentation left me thinking of that parameter as a
function object, but you can indeed pass "free" directly.