Discussion:
Problem in std::string, or unsupported mix of options.
(too old to reply)
James Kanze
2010-02-12 19:37:12 UTC
Permalink
I'm getting a very strange crash with the following program:

--------------------IteratorProblem.cpp------------------
typedef std::string::const_iterator text_iterator;

std::string r(std::string const& orig, std::string const& before,
std::string const& after)
{
std::string retval;
text_iterator current = orig.begin();
text_iterator end = orig.end();
text_iterator next = std::search(current, end, before.begin(),
before.end());
while (next != end)
{
retval.insert(retval.end(), current, next);
retval.insert(retval.end(), after.begin(), after.end());
current = next + before.size();
next = std::search(current, end, before.begin(),
before.end());
}
retval.insert(retval.end(), current, next);
return retval;
}

void r(std::string* text, std::string const& before, std::string
const& after)
{
std::string retval(r(*text, before, after));
text->swap(retval);
}

std::string f()
{
return r("double", "@", "");
}

std::string g(bool b,std::string const& a, std::string const& c)
{
return (b ? "Failed<" + a + '>' : a) + ' ' + c + ';';
}

std::string h()
{
std::string ret(g(true, f(), "x"));
r(&ret, ">>", "> >");
return ret;
}

int
main()
{
std::cout << "before:" << std::endl;
std::string s(h());
std::cout << "after: out = \"" << s << '\"' << std::endl;
return 0;
}
--------------------IteratorProblem.cpp------------------

The problem depends on the compiler options; for various
reasons, we are using
/D "_DEBUG" /D "_HAS_ITERATOR_DEBUGGING=0" /EHa /MDd /Zi
in debug mode, and this triggers the crash---if drop the
_HAS_ITERATOR_DEBUGGING line (so it defaults to 1 with _DEBUG),
everything works (except that the real code we are trying to
debug runs too slowly to make this acceptable). The funny thing
is, if I change the first parameter of g to false (in the call
in h), it also works; the problem only occurs if the final
string in r is greater than 15 characters (the small string
optimization buffer, and the capacity of a newly constructed
string). Checking in the debugger, it seems that despite the
_HAS_ITERATOR_DEBUGGING=0, some of the iterators are chaining in
with the owning string, but the chain is broken.

Given the somewhat special conditions required to trigger this
(a reallocation is necessary in "retval"), I suspect an error
somewhere in the library, but it's also possible that Microsoft
forbids this combination of options. (I can imagine cases where
it's likely to cause problems, but the library code seems to
take steps to guard against it.) So which is it? (And if it is
an illegal combination of options, where is this documented?)

--
James Kanze
Igor Tandetnik
2010-02-12 20:05:15 UTC
Permalink
Post by James Kanze
The problem depends on the compiler options; for various
reasons, we are using
/D "_DEBUG" /D "_HAS_ITERATOR_DEBUGGING=0" /EHa /MDd /Zi
in debug mode, and this triggers the crash---if drop the
_HAS_ITERATOR_DEBUGGING line (so it defaults to 1 with _DEBUG),
everything works
http://blogs.msdn.com/vcblog/archive/2007/02/26/stl-destructor-of-bugs.aspx

Search for "swap". Apparently, you should disable both _HAS_ITERATOR_DEBUGGING and _SECURE_SCL.
--
With best wishes,
Igor Tandetnik

With sufficient thrust, pigs fly just fine. However, this is not necessarily a good idea. It is hard to be sure where they are going to land, and it could be dangerous sitting under them as they fly overhead. -- RFC 1925
Stephan T. Lavavej [MSFT]
2010-02-12 22:05:23 UTC
Permalink
This bug, which is thoroughly fixed in VC10, is different from the Swap Bug,
which was fixed in VC9 RTM.

Compiling with /MDd /D_HAS_ITERATOR_DEBUGGING=0 /D_SECURE_SCL=0 won't help
here (I've verified that it still crashes). The underlying problem is that
before VC10, std::string was separately compiled into msvcp90d.dll, which
was built with HID=1 SCL=1. This has a really hard time talking to user
code built with different options. VC8 and VC9 had what I refer to as
"broken library magic" that makes this work in most cases. Unfortunately,
most isn't all, and there were bugs in this in all of VC8 RTM, VC8 SP1, VC9
RTM, and VC9 SP1. Previously, the only bugs in VC9 SP1 that I were aware of
involved ostringstream str(), but this bug operates on the same principle
(operator+() returns a string by value, and that's separately compiled into
the DLL - you really don't want to know all of the gory details).

You should upgrade to VC10 as soon as possible (we've got a "go live"
Release Candidate that was just released). You should certainly be running
VC9 SP1 now, in order to avoid the HID/SCL bugs we've already fixed. As a
workaround, my experiments indicate that avoiding op+() will do the trick -
you can instead say string ret; ret += stuff; ret += more stuff; return ret;
and that will create the string on your side instead of in the DLL.

(The broken library magic was written long before my time, but I was
personally responsible for ripping it out in VC10.)

STL
Post by James Kanze
The problem depends on the compiler options; for various
reasons, we are using
/D "_DEBUG" /D "_HAS_ITERATOR_DEBUGGING=0" /EHa /MDd /Zi
in debug mode, and this triggers the crash---if drop the
_HAS_ITERATOR_DEBUGGING line (so it defaults to 1 with _DEBUG),
everything works
http://blogs.msdn.com/vcblog/archive/2007/02/26/stl-destructor-of-bugs.aspx

Search for "swap". Apparently, you should disable both
_HAS_ITERATOR_DEBUGGING and _SECURE_SCL.
--
With best wishes,
Igor Tandetnik

With sufficient thrust, pigs fly just fine. However, this is not necessarily
a good idea. It is hard to be sure where they are going to land, and it
could be dangerous sitting under them as they fly overhead. -- RFC 1925
James Kanze
2010-02-15 09:10:34 UTC
Permalink
Post by Stephan T. Lavavej [MSFT]
This bug, which is thoroughly fixed in VC10, is different from
the Swap Bug, which was fixed in VC9 RTM.
Compiling with /MDd /D_HAS_ITERATOR_DEBUGGING=0
/D_SECURE_SCL=0 won't help here (I've verified that it still
crashes).
I was going to answer Igor that in the actual code, we had
_SECURE_SCL set to 0 as well. I'd dropped it in the example,
since changing it didn't change anything.
Post by Stephan T. Lavavej [MSFT]
The underlying problem is that before VC10, std::string was
separately compiled into msvcp90d.dll, which was built with
HID=1 SCL=1. This has a really hard time talking to user code
built with different options. VC8 and VC9 had what I refer to
as "broken library magic" that makes this work in most cases.
Unfortunately, most isn't all, and there were bugs in this in
all of VC8 RTM, VC8 SP1, VC9 RTM, and VC9 SP1. Previously,
the only bugs in VC9 SP1 that I were aware of involved
ostringstream str(), but this bug operates on the same
principle (operator+() returns a string by value, and that's
separately compiled into the DLL - you really don't want to
know all of the gory details).
I suspected something along these lines. I've seen quite a few
problems with g++ due to different libraries being compiled with
or without _GLIBCXX_DEBUG (which is more or less the equivalent
of _HAS_ITERATOR_DEBUGGING), and when tracking the problem in
the debugger, I found the debugger showing msvcp90d.dll rather
than my executable. And while I'm not really too sure how the
debugger decides what to display here (this was in the stack
window), the addresses in the disassembled code were in a
different area than those in my own code, which also suggests a
different DLL.
Post by Stephan T. Lavavej [MSFT]
You should upgrade to VC10 as soon as possible (we've got a
"go live" Release Candidate that was just released).
I'll inform my management, but they make the decisions, not I.
Post by Stephan T. Lavavej [MSFT]
You
should certainly be running VC9 SP1 now, in order to avoid the
HID/SCL bugs we've already fixed.
We're still using VC8, but just out of curiousity, how do you
determine the patch level of the compiler? If I just invoke cl
at the command line, I get:
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version
14.00.50727.762 for 80x86
Which doesn't even tell *me* it's VC8. (I assume it's VC8
because that's what my boss told me it was. Also, the path is
through c:\Program Files\Microsoft Visual Studio 8.) And I
don't even get to see this (or the command line) when I invoke
it from Visual Studios.
Post by Stephan T. Lavavej [MSFT]
As a workaround, my experiments indicate that avoiding op+()
will do the trick - you can instead say string ret; ret +=
stuff; ret += more stuff; return ret; and that will create the
string on your side instead of in the DLL.
OK. I'll give it a try.

It still surprises me, because we have a significant code base
compiled like this, using the same options, and the problem has
never appeared for others. Most of the other code is in DLL's,
which plug in to other software (Excel, etc.). Could that make
a difference?

--
James Kanze
David Wilkinson
2010-02-15 15:32:35 UTC
Permalink
Post by James Kanze
Post by Stephan T. Lavavej [MSFT]
You should upgrade to VC10 as soon as possible (we've got a
"go live" Release Candidate that was just released).
I'll inform my management, but they make the decisions, not I.
Post by Stephan T. Lavavej [MSFT]
You
should certainly be running VC9 SP1 now, in order to avoid the
HID/SCL bugs we've already fixed.
We're still using VC8, but just out of curiousity, how do you
determine the patch level of the compiler? If I just invoke cl
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version
14.00.50727.762 for 80x86
Which doesn't even tell *me* it's VC8. (I assume it's VC8
because that's what my boss told me it was. Also, the path is
through c:\Program Files\Microsoft Visual Studio 8.) And I
don't even get to see this (or the command line) when I invoke
it from Visual Studios.
Compiler versions:

1100 Visual Studio 5.0 (VC5)
1200 Visual Studio 6.0 (VC6)
1300 Visual Studio .NET 2002 (VC7)
1310 Visual Studio .NET 2003 (VC7.1)
1400 Visual Studio 2005 (VC8)
1500 Visual Studio 2008 (VC9)
1600 Visual Studio 2010 (VC10)
--
David Wilkinson
Visual C++ MVP
Stephan T. Lavavej [MSFT]
2010-02-15 18:49:02 UTC
Permalink
[James]
I've seen quite a few problems with g++ due to different libraries
being compiled with or without _GLIBCXX_DEBUG (which is
more or less the equivalent of _HAS_ITERATOR_DEBUGGING)
The other nice thing about VC10 is that it detects mismatch at link time.
We're still using VC8, but just out of curiousity, how do you
determine the patch level of the compiler?
David has provided the correct decoder ring. It's complicated because VS
2005 contains the 8th version of the Visual C++ compiler, which is the 14th
version of the C++ compiler.
14.00.50727.762 for 80x86
Which doesn't even tell *me* it's VC8.
That's VC8 SP1.
And I don't even get to see this (or the command line) when I invoke
it from Visual Studios.
VS's Help About lists its version and service pack level, but the compiler's
full version is more useful.

STL
Post by James Kanze
Post by Stephan T. Lavavej [MSFT]
You should upgrade to VC10 as soon as possible (we've got a
"go live" Release Candidate that was just released).
I'll inform my management, but they make the decisions, not I.
Post by Stephan T. Lavavej [MSFT]
You
should certainly be running VC9 SP1 now, in order to avoid the
HID/SCL bugs we've already fixed.
We're still using VC8, but just out of curiousity, how do you
determine the patch level of the compiler? If I just invoke cl
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version
14.00.50727.762 for 80x86
Which doesn't even tell *me* it's VC8. (I assume it's VC8
because that's what my boss told me it was. Also, the path is
through c:\Program Files\Microsoft Visual Studio 8.) And I
don't even get to see this (or the command line) when I invoke
it from Visual Studios.
1100 Visual Studio 5.0 (VC5)
1200 Visual Studio 6.0 (VC6)
1300 Visual Studio .NET 2002 (VC7)
1310 Visual Studio .NET 2003 (VC7.1)
1400 Visual Studio 2005 (VC8)
1500 Visual Studio 2008 (VC9)
1600 Visual Studio 2010 (VC10)
--
David Wilkinson
Visual C++ MVP
Jeff Flinn
2010-02-16 13:06:14 UTC
Permalink
Hi Steven,
Post by Stephan T. Lavavej [MSFT]
You should upgrade to VC10 as soon as possible (we've got a "go live"
Release Candidate that was just released). You should certainly be
running VC9 SP1 now, in order to avoid the HID/SCL bugs we've already
fixed. As a workaround, my experiments indicate that avoiding op+()
will do the trick - you can instead say string ret; ret += stuff; ret +=
more stuff; return ret; and that will create the string on your side
instead of in the DLL.
(The broken library magic was written long before my time, but I was
personally responsible for ripping it out in VC10.)
STL
Is there a static msvcpxx.lib? If so what project property allows it's use?

Thanks, Jeff
Igor Tandetnik
2010-02-16 13:11:42 UTC
Permalink
Post by Jeff Flinn
Is there a static msvcpxx.lib? If so what project property allows it's use?
/MT[d], aka Project | Properties | C/C++ | Code Generation | Runtime Library = Multhi-Threaded [Debug]
--
With best wishes,
Igor Tandetnik

With sufficient thrust, pigs fly just fine. However, this is not necessarily a good idea. It is hard to be sure where they are going to land, and it could be dangerous sitting under them as they fly overhead. -- RFC 1925
Loading...