Why is reference from unused function required?

[John N.]

I have stumbled across a problem involving linking to a static library
that I was hoping someone could help me with.

I want to create a static library which contains various methods that
I will use from other code. In my simple example case, I try to call
one of the two library methods from main(). Everything compiles
correctly. However, when linking, I get the link error shown below.

The source files (Methods.h, Methods.cpp, Tester.cpp) for my simple
example are typed out at the end of this post.

“Methods.h” and “Methods.cpp” are used to build a static library named
“support library.lib”, while “Tester.cpp” is the main() which uses one
of the two methods (method1) defined in “Methods.cpp”.

The other method in the static library (method2) in “Methods.cpp” is
not referenced by main(). It in turn references a method
(methodDoesNotExist) that is declared in “Methods.h”, but is not
defined in “Methods.cpp”. Thus, the static library compiles and is
built with no errors, yet when I go to link it with the main() I get
the link error shown below.

Why does the linker give me this link error since I am not trying to
use method2?

One way to “fix” the problem would be to create separate .cpp/h files
for each of the methods and then use all those files to create the
static library.

Is there any way to keep all the methods in the same file (for ease of
maintenance), yet have the linker only resolve references that are
used?

Thank you.
John N.

Small example source files begin here:

// Methods.h
//*************************************************
#include <string>
#include <istream>

const std::string method1( std::istream& stream_ );
const std::string method2( std::istream& stream_ );
const std::string methodDoesNotExist();
//*************************************************

// Methods.cpp
//**********************************
#include <string> // std::string
#include <istream> // std::istream
#include "Methods.h" // ::Methods_xxx

const std::string
method1( std::istream& stream_ )
{
return "from method1";
}

const std::string
method2( std::istream& stream_ )
{
return methodDoesNotExist();
}
//**********************************

// Tester.cpp
//****************************************************
#include "Methods.h"
#include <fstream>
#include <iostream> // std::cout
#include <string>

int main(){
std::ifstream stream;
stream.open("testerParse.txt");

std::cout << ::method1( stream ) << std::endl;

stream.close();
char c;
std::cin >> c;
}
//****************************************************

Link Error:

------ Build started: Project: support library, Configuration: Debug
Win32 ------
Compiling...
Methods.cpp
Creating library...
Build log was saved at "file://c:\Test\BuildLog.htm"
support library - 0 error(s), 0 warning(s)
------ Build started: Project: testerParse, Configuration: Debug Win32
------
Linking...
support library.lib(Methods.obj) : error LNK2019: unresolved external
symbol "class std::basic_string<char,struct
std::char_traits<char>,class std::allocator<char> > const __cdecl
methodDoesNotExist(void)" (?methodDoesNotExist@@YA?BV?$***@DU?
$***@D@std@@V?$***@D@2@@std@@XZ) referenced in function
"class std::basic_string<char,struct std::char_traits<char>,class
std::allocator<char> > const __cdecl method2(class
std::basic_istream<char,struct std::char_traits<char> > &)" (?
method2@@YA?BV?$***@DU?$***@D@std@@V?
$***@D@2@@std@@AAV?$***@DU?$***@D@std@@@2@@Z)
C:\Test\testerParse.exe : fatal error LNK1120: 1 unresolved externals
Build log was saved at "file://c:\Test\BuildLog.htm"
testerParse - 2 error(s), 0 warning(s)
========== Build: 1 succeeded, 1 failed, 0 up-to-date, 0 skipped
==========

[David Wilkinson]

Post by John N.
I have stumbled across a problem involving linking to a static library
that I was hoping someone could help me with.
I want to create a static library which contains various methods that
I will use from other code. In my simple example case, I try to call
one of the two library methods from main(). Everything compiles
correctly. However, when linking, I get the link error shown below.
The source files (Methods.h, Methods.cpp, Tester.cpp) for my simple
example are typed out at the end of this post.
“Methods.h” and “Methods.cpp” are used to build a static library named
“support library.lib”, while “Tester.cpp” is the main() which uses one
of the two methods (method1) defined in “Methods.cpp”.
The other method in the static library (method2) in “Methods.cpp” is
not referenced by main(). It in turn references a method
(methodDoesNotExist) that is declared in “Methods.h”, but is not
defined in “Methods.cpp”. Thus, the static library compiles and is
built with no errors, yet when I go to link it with the main() I get
the link error shown below.
Why does the linker give me this link error since I am not trying to
use method2?
One way to “fix” the problem would be to create separate .cpp/h files
for each of the methods and then use all those files to create the
static library.
Is there any way to keep all the methods in the same file (for ease of
maintenance), yet have the linker only resolve references that are
used?
Thank you.
John N.
// Methods.h
//*************************************************
#include <string>
#include <istream>
const std::string method1( std::istream& stream_ );
const std::string method2( std::istream& stream_ );
const std::string methodDoesNotExist();
//*************************************************
// Methods.cpp
//**********************************
#include <string> // std::string
#include <istream> // std::istream
#include "Methods.h" // ::Methods_xxx
const std::string
method1( std::istream& stream_ )
{
return "from method1";
}
const std::string
method2( std::istream& stream_ )
{
return methodDoesNotExist();
}
//**********************************
// Tester.cpp
//****************************************************
#include "Methods.h"
#include <fstream>
#include <iostream> // std::cout
#include <string>
int main(){
std::ifstream stream;
stream.open("testerParse.txt");
std::cout << ::method1( stream ) << std::endl;
stream.close();
char c;
std::cin >> c;
}
//****************************************************
------ Build started: Project: support library, Configuration: Debug
Win32 ------
Compiling...
Methods.cpp
Creating library...
Build log was saved at "file://c:\Test\BuildLog.htm"
support library - 0 error(s), 0 warning(s)
------ Build started: Project: testerParse, Configuration: Debug Win32
------
Linking...
support library.lib(Methods.obj) : error LNK2019: unresolved external
symbol "class std::basic_string<char,struct
std::char_traits<char>,class std::allocator<char> > const __cdecl
"class std::basic_string<char,struct std::char_traits<char>,class
std::allocator<char> > const __cdecl method2(class
std::basic_istream<char,struct std::char_traits<char> > &)" (?
C:\Test\testerParse.exe : fatal error LNK1120: 1 unresolved externals
Build log was saved at "file://c:\Test\BuildLog.htm"
testerParse - 2 error(s), 0 warning(s)
========== Build: 1 succeeded, 1 failed, 0 up-to-date, 0 skipped
==========

I am no expert at this, but I think you might take a look at

Function-Level Linking (/Gy switch in VC++)

[John N.]

Post by David Wilkinson

Post by John N.
I have stumbled across a problem involving linking to a static library
that I was hoping someone could help me with.
I want to create a static library which contains various methods that
I will use from other code. In my simple example case, I try to call
one of the two library methods from main(). Everything compiles
correctly. However, when linking, I get the link error shown below.
The source files (Methods.h, Methods.cpp, Tester.cpp) for my simple
example are typed out at the end of this post.
“Methods.h” and “Methods.cpp” are used to build a static library named
“support library.lib”, while “Tester.cpp” is the main() which uses one
of the two methods (method1) defined in “Methods.cpp”.
The other method in the static library (method2) in “Methods.cpp” is
not referenced by main(). It in turn references a method
(methodDoesNotExist) that is declared in “Methods.h”, but is not
defined in “Methods.cpp”. Thus, the static library compiles and is
built with no errors, yet when I go to link it with the main() I get
the link error shown below.
Why does the linker give me this link error since I am not trying to
use method2?
One way to “fix” the problem would be to create separate .cpp/h files
for each of the methods and then use all those files to create the
static library.
Is there any way to keep all the methods in the same file (for ease of
maintenance), yet have the linker only resolve references that are
used?
Thank you.
John N.
// Methods.h
//*************************************************
#include <string>
#include <istream>
const std::string method1( std::istream& stream_ );
const std::string method2( std::istream& stream_ );
const std::string methodDoesNotExist();
//*************************************************
// Methods.cpp
//**********************************
#include <string> // std::string
#include <istream> // std::istream
#include "Methods.h" // ::Methods_xxx
const std::string
method1( std::istream& stream_ )
{
   return "from method1";
}
const std::string
method2( std::istream& stream_ )
{
   return methodDoesNotExist();
}
//**********************************
// Tester.cpp
//****************************************************
#include "Methods.h"
#include <fstream>
#include <iostream> // std::cout
#include <string>
int main(){
   std::ifstream stream;
   stream.open("testerParse.txt");
   std::cout << ::method1( stream ) << std::endl;
   stream.close();
   char c;
   std::cin >> c;
}
//****************************************************
------ Build started: Project: support library, Configuration: Debug
Win32 ------
Compiling...
Methods.cpp
Creating library...
Build log was saved at "file://c:\Test\BuildLog.htm"
support library - 0 error(s), 0 warning(s)
------ Build started: Project: testerParse, Configuration: Debug Win32
------
Linking...
support library.lib(Methods.obj) : error LNK2019: unresolved external
symbol "class std::basic_string<char,struct
std::char_traits<char>,class std::allocator<char> > const __cdecl
"class std::basic_string<char,struct std::char_traits<char>,class
std::allocator<char> > const __cdecl method2(class
std::basic_istream<char,struct std::char_traits<char> > &)" (?
C:\Test\testerParse.exe : fatal error LNK1120: 1 unresolved externals
Build log was saved at "file://c:\Test\BuildLog.htm"
testerParse - 2 error(s), 0 warning(s)
========== Build: 1 succeeded, 1 failed, 0 up-to-date, 0 skipped
==========

I am no expert at this, but I think you might take a look at
Function-Level Linking (/Gy switch in VC++)
--
David Wilkinson
Visual C++ MVP

David,

Thank you for the response.

I tried changing the option in my solution (for both the .lib
and .exe) under "Project Properties->Code Generation->Enable Function-
Level Linking->/Gy (Yes)", however, I still get the same linker error
(as seen in first post) when I build the project (in both debug and
release mode, just in case that might make any difference).

Thanks,
John

[Ian Harvey]

Post by John N.

Post by David Wilkinson

Post by John N.
Why does the linker give me this link error since I am not trying to
use method2?

...

Post by John N.

Post by David Wilkinson

Post by John N.
One way to “fix” the problem would be to create separate .cpp/h files
for each of the methods and then use all those files to create the
static library.
Is there any way to keep all the methods in the same file (for ease of
maintenance), yet have the linker only resolve references that are
used?

I am no expert at this, but I think you might take a look at
Function-Level Linking (/Gy switch in VC++)
--
David Wilkinson
Visual C++ MVP

David,
Thank you for the response.
I tried changing the option in my solution (for both the .lib
and .exe) under "Project Properties->Code Generation->Enable Function-
Level Linking->/Gy (Yes)", however, I still get the same linker error
(as seen in first post) when I build the project (in both debug and
release mode, just in case that might make any difference).
Thanks,
John

The linker (proper) doesn't necessarily know what will or will not be
*called* inside the obj file - it only deals with resolving references
to things.

Perhaps combine /Gy with linker options /FORCE:UNRESOLVED and /OPT:REF.
You'll still get an error, but the executable will (probably) work.

You can keep the one header file, but the conventional approach of
having separate cpp files (that result in individual obj files), that
are then combined together into a static library might actually be less
long term pain, given some of the issues that might arise from /FORCE:xxx.

[John N.]

Post by Ian Harvey

Post by John N.

Post by David Wilkinson

Post by John N.
Why does the linker give me this link error since I am not trying to
use method2?

...

Post by John N.

Post by David Wilkinson

Post by John N.
One way to “fix” the problem would be to create separate .cpp/h files
for each of the methods and then use all those files to create the
static library.
Is there any way to keep all the methods in the same file (for ease of
maintenance), yet have the linker only resolve references that are
used?

I am no expert at this, but I think you might take a look at
Function-Level Linking (/Gy switch in VC++)
--
David Wilkinson
Visual C++ MVP

David,
Thank you for the response.
I tried changing the option in my solution (for both the .lib
and .exe) under "Project Properties->Code Generation->Enable Function-
Level Linking->/Gy (Yes)", however, I still get the same linker error
(as seen in first post) when I build the project (in both debug and
release mode, just in case that might make any difference).
Thanks,
John

The linker (proper) doesn't necessarily know what will or will not be
*called* inside the obj file - it only deals with resolving references
to things.
Perhaps combine /Gy with linker options /FORCE:UNRESOLVED and /OPT:REF.
  You'll still get an error, but the executable will (probably) work.
You can keep the one header file, but the conventional approach of
having separate cpp files (that result in individual obj files), that
are then combined together into a static library might actually be less
long term pain, given some of the issues that might arise from /FORCE:xxx.

Thanks Ian.

Post by Ian Harvey
the conventional approach of having separate cpp files (that result in individual obj files)

I was actually hoping there was a means for the compiler to package
things in the obj file so that separate functions are separate (the
obj would effectively be a static library with an obj name). I guess
this does not exist.

Does this mean that an entire C++ class is brought into an exe even if
only a small percentage of it is used?

[Tim Roberts]

Post by John N.
I was actually hoping there was a means for the compiler to package
things in the obj file so that separate functions are separate (the
obj would effectively be a static library with an obj name). I guess
this does not exist.

No. The unit of linkage is an object file, not a function. There are very
good reasons for this. "Functions" are not a linker concept. The division
of code into units called "functions" is simply an arbitrary convention of
whatever compiler you're using. In the object file, you just have a
sequence of machine language instructions that happens to have public names
attached every now and then.

In many assembler programs, for example, the code is not so neatly
packaged. Functions might flow into other functions, or jump into other
parts of the object file.

Now, it's true that the linker can usually tell whether an object file was
created by a C compiler. Knowing that, and knowing the internals of the
compiler, it can make certain assumptions about the organization of the
code. If you specify linker optimizations with the /OPT flag, it will
attempt to do that.

Post by John N.
Does this mean that an entire C++ class is brought into an exe even if
only a small percentage of it is used?

Generally, if the class is implemented in a single source file, yes.

Note that this MUST be true for the virtual functions in a class. The
virtual function table for a class has a pointer to every virtual function
it implements. Thus, there will ALWAYS be a reference to every virtual
function. There's no way to tell whether a particular virtual function
will ever be called without simulating the executing the code, and Turing
gets in the way of that.

[John N.]

Post by John N.
I was actually hoping there was a means for the compiler to package
things in the obj file so that separate functions are separate (the
obj would effectively be a static library with an obj name). I guess
this does not exist.

No.  The unit of linkage is an object file, not a function.  There are very
good reasons for this.  "Functions" are not a linker concept.  The division
of code into units called "functions" is simply an arbitrary convention of
whatever compiler you're using.  In the object file, you just have a
sequence of machine language instructions that happens to have public names
attached every now and then.
In many assembler programs, for example, the code is not so neatly
packaged.  Functions might flow into other functions, or jump into other
parts of the object file.
Now, it's true that the linker can usually tell whether an object file was
created by a C compiler.  Knowing that, and knowing the internals of the
compiler, it can make certain assumptions about the organization of the
code.  If you specify linker optimizations with the /OPT flag, it will
attempt to do that.

Post by John N.
Does this mean that an entire C++ class is brought into an exe even if
only a small percentage of it is used?

Generally, if the class is implemented in a single source file, yes.
Note that this MUST be true for the virtual functions in a class.  The
virtual function table for a class has a pointer to every virtual function
it implements.  Thus, there will ALWAYS be a reference to every virtual
function.  There's no way to tell whether a particular virtual function
will ever be called without simulating the executing the code, and Turing
gets in the way of that.
--
Providenza & Boekelheide, Inc.

Tim,

Thank you for the informative post.

There's no way to tell whether a particular virtual function
will ever be called without simulating the executing the code, and Turing
gets in the way of that.

If a virtual method is not referenced in the entire executable, then
why is it required?

[Tamas Demjen]

Post by John N.
If a virtual method is not referenced in the entire executable, then
why is it required?

The compiler doesn't know if it's going to be called, therefore it has
no choice but to put it into the vtable. And once it's there, the
function must be linked.

Certain optimizations could only be done at link time. Traditionally
linkers weren't smart enough to make significant modifications to the
machine code, so link-time optimizations were ruled out.

In recent versions of Visual Studio whole program optimization is
actually available as an option:
http://msdn.microsoft.com/en-us/library/0zza0de8%28VS.80%29.aspx

This requires the compiler to produce an intermediate code, and the
linker is responsible for generating the final machine code, so it has a
chance to perform a bunch of optimizations that the compiler couldn't
see at the object file level. This way it can remove unused functions,
unused vtable entries, and decide to make virtual functions non-virtual
when possible (when they're not called via a base pointer, they don't
really need to be virtual).

Another example is Java, where every function is virtual by default. It
is the responsibility of the JIT compiler to figure out which functions
actually require dynamic dispatching. That decision is not part of the
intermediate byte-code. Virtual functions can even be inlined when the
type is known at JIT-time.

Tom

[Tim Roberts]

Post by John N.

Post by Tim Roberts
There's no way to tell whether a particular virtual function
will ever be called without simulating the executing the code, and Turing
gets in the way of that.

If a virtual method is not referenced in the entire executable, then
why is it required?

Consider the assembler code for calling a non-virtual method:

mov ecx, [ObjectInstance]
call CMyClass::GetObjectData

There is an explicit reference to the method name. The linker sees that
reference, and knows that someone needs that function. A function that is
not called is never referenced, and can safely be removed;

Now consider calling a virtual method:

CMyClass_vtable:
dd CMyClass::QueryInterface
dd CMyClass::AddRef
dd CMyClass::Release
dd CMyClass::GetObjectData
...
mov ecx, [ObjectInstance]
mov eax, [ecx] ; Get address of vtable
call dword ptr [eax+12] ; call 4th entry

The call goes indirectly through the function pointers in the vtable. The
linker sees explicit references to ALL of the virtual function names.
There's no way for the linker to know that some of them are not actually
used in the code.