PGI Linking Questions

Can you give a general overview of linker problems?

Linkers create executable files from object files and libraries, by binding external references of calling programs with the entry points of the called routines. The external references and entry points should be identical symbols for the link to succeed.

Object files are put into an executable file and are linked to any references made to any objects already linked. Library archives (libxxx.a) are collections of objects where the object is extracted from the library and put into the executable. Shared libraries (libxxx.so) are only partially linked in the linker, and at runtime the shared library present will be read and the routine extracted and run. This allows users to link with one library in the linker, and link the actual routine on the platform at runtime.

The most common linker problems are:

How do I fix a ": undefined reference to `f__xargc'" problem?

This is a common error when linking g77 compiled programs with the PGI compiler-driver. 

> Linking:
> /usr/local/mpich/build/LINUX/ch_p4/lib/libmpich.a(farg.o): In function
> `mpir_iargc__':
> farg.o(.text+0x8): undefined reference to `f__xargc'

Both xargc and xargv are g77 global variables which represent the Fortran equivalents of the C variables argc and argv. The file, farg.f, contains references to the functions iargc() and getarg(). When g77 is used to compile farg.f, these variables are used by g77-generated code for the iargc() and getarg() functions. These variables are defined when there is a g77-compiled main program. pgf77 also has global variables for the same purpose and are named __argc_save an __argv_save.

There are a couple of work-arounds:

  1. Add a C function which defines and assigns xargc and xargv and add a call to the C function from the Fortran main program. For example: 

        zarg.c:
        int    xargc;
        char **xargv;
        extern int    __argc_save;
        extern char **__argv_save;
        void zarg_() {          /* call zarg() from Fortran */
            xargc = __argc_save;
            xargv = __argv_save;
        }

  2. Recompile the file farg.f with pgf77 -c and add the object to the link before ‑lmpich. If the second underscore is needed for global names, remember to include the second underscore option. The source for farg.f is: 

       integer function mpir_iargc()
      mpir_iargc = iargc()
      return
      end
      subroutine mpir_getarg( i, s )
      integer       i
      character*(*) s
      call getarg(i,s)
      return
      end

How do I solve the underscore problems?

There are two common underscore problems:

  1. gcc produces a second underscore. Try compiling with ‑Msecond_underscore

  2. Calling C routines from Fortran. This is discussed in Chapter 10 of the PGI User's Guide. If you do not want to create a wrapper function, you can use the directive 

    !$PRAGMA C (rout1, rout2, etc)

    to direct the Fortran compilers to not underscore the symbols for rout1 and rout2 and so on.

How do I tell the linker to ignore unresolved references?

PGI uses the linker that is part of the development tools installed on your system (e.g. GCC, Visual Studio, Xcode, etc.). On Linux, this is usually ld. The ld option ‑noinhibit-exec will usually produce an executable file even if there are linking errors. To pass switches to the linker, use the ‑Wl, switch along with the linker switch. For example: 

%cat x.c
main() {
         while (0)
         {
              foo();
         }
}

pgcc -o x x.c -Wl,-noinhibit-exec

Can I use the Intel MKL with PGI compilers and if so, how do I do it?

The Intel Math Kernel Library is a library of math functions highly optimized for Intel's line of microprocessors. With release 10.0 and later of the MKL, Intel adopted a layered architecture. This modular approach separates the MKL into four distinct functional layers:

All compiler-specific run-time library references are now isolated in the threading layer. In Release 10.1 and above of the MKL, Intel included threading libraries for PGI compilers. Following are the steps required to use the MKL with multi-threaded OpenMP programs compiled with PGI.

Using MKL with PGI Workstation and PGI Server
For PGI Workstation and PGI Server class products on Linux, Mac OS X and Windows, add one of the MKL supplied libraries for each of the first three layers (interface, threading, computation) to your linker command line. Use the ‑mp option on the linker command line as well to ensure the multi-threaded PGI runtime library is used. Also, check that you are linking in the appropriate PGI threading library (either static or dynamic) from the MKL.

Here is an example of compiling an OpenMP multi-threaded version of a simple matmul program (pgi_mm.F) with PGI Fortran (pgfortran) on Linux, and linking in the MKL.

  1. Compile with OpenMP directive processing enabled (‑mp) 

     > pgfortran -mp -Minfo -fast -c ../pgi_mm.F

  2. Link the three MKL supplied libraries with the compiled object file. The ‑mp option will link in the PGI multi-threaded runtime library. 

     % pgf90 -mp -Minfo -o mm_mp pgi_mm.o ../dclock_64.o -R/opt/mkl/lib/em64t -L/opt/mkl/lib/em64t 
   -lmkl_intel_lp64 -lmkl_pgi_thread -lmkl_core

Note: dclock_64 is a clock function used in the matmul program that returns elapsed time in seconds.

Here is the same souce code example compiled using the sequential non-threaded version of the MKL threading library.

  1. Compile serially (no ‑mp option) 

     % pgf90 -Minfo -fast -c ../pgi_mm.F

  2. Link the three MKL supplied libraries with the compiled object file. No ‑mp option links in the PGI serial runtime library. 

     % pgf90 -Minfo -o mm pgi_mm.o ../dclock_64.o -R/opt/mkl/lib/em64t -L/opt/mkl/lib/em64t 
   -lmkl_intel_lp64 -lmkl_sequential -lmkl_core

Determining which library to use for optimum performance involves a number of factors including array sizes, the choice of single or double precision arithmetic and the number of processors and/or cores available. Experimentation is the only sure way to know what's best for your situation.

Using MKL with PGI Visual Fortran
To use MKL with PGI Visual Fortran®, follow the directions for configuring MKL for Intel Visual Fortran in Section 4 of the MKL for Windows User's Guide. Substitute the appropriate mkl_pgi_thread* for the mkl_intel_thread* library as an Additional Dependency in the Configuration Properties > Linker > Input section of your project's Property Pages window. Enable the Process OpenMP Directives option through the Configuration Properties > Fortran > Language section as well to ensure the multi-treaded runtime layer is included.

See the Intel MKL documentation page for more information.

How can I use the Linux LIBRARY_PATH and LDFLAGS environment with the PGI compilers?

The environment variable LIBRARY_PATH is used by some gcc drivers to add directories to the library search paths. This can be done with a few lines in the siterc file in the PGI installation directory.

Create a siterc file with the following lines, or add the following lines to your existing siterc file; an alternative is to add these lines to a .mypgccrc, .mypgfortranrc, .mypgf77rc or .mypgCCrc files in your home directory. 

# get the value of the environment variable LIBRARY_PATH
variable LIBRARY_PATH is environment(LD_LIBRARY_PATH);

# split this value at colons, separate by -L, prepend 1st one by -L
variable library_path is
default($if($LIBRARY_PATH,-L$replace($LIBRARY_PATH,":", -L)));

# add the -L arguments to the link line
append LDLIBARGS=$library_path;

What do I do when the linker complains "ld: cannot open -lgcc: No such file or directory"?

This happens when the compiler can't find where gcc has been installed (the Linux system libraries are found in the gcc area).

This is usually the result of

  1. gcc has not been installed, has been moved or removed.
  2. gcc has been updated since the most recent installation.

The usual remedy is to reinstall the compilers, making sure that the gcc environment is set up properly (try compiling and executing hello.c with gcc). This will ensure that the PGI drivers point to the proper areas for locating the system libraries.

To make sure things are correct, try compiling hello.c with both gcc and pgcc. 

gcc -v hello.c

pgcc -v hello.c

Analyze the dialogue created and determine if the gcc libs being brought in are identical.

How do I generate a link map?

Memory maps are produced by your system's linker. PGI compilers can pass switches directly to linkers by using the ‑Wl, switch on the Linux command line. Linkers for most operating systems have a similar option.

For example, if ‑Map mapfilename creates a memory map in the file called mapfilename using the Linux ld linker, then the compilers can create the same map file using 

pgfortran -o x x.f -Wl,-Map,mapfilename

Click me