PGI Linking Questions

How do I fix a ": undefined reference to `f__xargc'" problem?

I'm getting the following error when I link g77 compiled programs with your compiler-driver 

> Linking:
> /usr/local/mpich/build/LINUX/ch_p4/lib/libmpich.a(farg.o): In function
> `mpir_iargc__':
> farg.o(.text+0x8): undefined reference to `f__xargc'

xargc & xargv are g77's global variables which represent the fortran equivalents of C's argc & argv. The file, farg.f, contains references to the functions iargc() and getarg(). When g77 is used to compile farg.f, these variables are used by g77-generated code for the iargc() and getarg() functions. These variables are defined when there is a g77-compiled main program. pgf77 also has global variables for the same purpose and are named __argc_save & __argv_save.

There are a couple of work-arounds:

  1. Add a C function which defines and assigns xarg & xargc and add a call to the C function from the fortran main program. For example: 

        zarg.c:
        int    xargc;
        char **xargv;
        extern int    __argc_save;
        extern char **__argv_save;
        void zarg_() {          /* call zarg() from fortran */
            xargc = __argc_save;
            xargv = __argv_save;
        }

  2. Recompile the file farg.f with 'pgf77 -c' and add the object to the link before -lmpich. If the second underscore is needed for global names, remember to include the second underscore option. The source for farg.f is: 

       integer function mpir_iargc()
      mpir_iargc = iargc()
      return
      end
      subroutine mpir_getarg( i, s )
      integer       i
      character*(*) s
      call getarg(i,s)
      return
      end

How do I solve the underscore problems?

There are two underscore problems that people run into

  1. gcc produces an second underscore. Try compiling with -Msecond_underscore

  2. Calling C routines from fortran. This is discussed in Chapter 10 of the PGI User's Guide . If you do not want to create a 'wrapper' function, you can use the directive 

    !$PRAGMA C (rout1, rout2, etc)

    to direct the fortran compilers to not underscore the symbols for rout1 and rout2 and so on.

How do I tell the linker to ignore unresolved references?

We use the linker that comes with your installation. This is usually ld, and the version we work with has a -noinhibit-exec switch. To pass switches to the linker, -Wl,switch will pass a literal switch.

Example: 

%cat x.c
main() {
         while (0)
         {
              foo();
         }
}

pgcc -o x x.c -Wl,-noinhibit-exec

produces a working executable.

Can I use the MKL with PGI compilers and if so, how do I do it?

The Intel Math Kernel Library is a library of math functions highly optimized for Intel's line of microprocessors. With release 10.0 and later of the MKL, Intel adopted a layered architecture. This modular approach separates the MKL into four distinct functional layers:

All compiler-specific run-time library references are now isolated in the threading layer. In Release 10.1 and above of the MKL, Intel included threading libraries for PGI compilers. Following are the steps required to use the MKL with multi-threaded OpenMP programs compiled with PGI.

Using MKL with PGI Workstation and PGI Server
For PGI Workstation and PGI Server class products on Linux, Mac OS X and Windows, add one of the MKL supplied libraries for each of the first three layers (interface, threading, computation) to your linker command line. Use the -mp option on the linker command line as well to ensure the multi-threaded PGI run-time library is used. Also, check that you are linking in the appropriate PGI threading library (either static or dynamic) from the MKL.

Here is an example of compiling an OpenMP multi-threaded version of a simple matmul program (pgi_mm.F) with PGI Fortran 95 (pgf95) on Linux, and linking in the MKL.

  1. Compile with OpenMP directive processing enabled (-mp) 

     > pgf90 -mp -Minfo -fast -c ../pgi_mm.F

  2. Link the three MKL supplied libraries with the compiled object file. The -mp option will link in the PGI multi-threaded run-time library. 

     % pgf90 -mp -Minfo -o mm_mp mm.o ../dclock_64.o -R/opt/mkl/lib/em64t -L/opt/mkl/lib/em64t 
   -lmkl_intel_lp64 -lmkl_pgi_thread -lmkl_core

Note: dclock_64 is a clock function used in the matmul program that returns elapsed time in seconds.

Here is the same souce code example compiled using the sequential non-threaded version of the MKL threading library.

  1. Compile serially (no -mp option) 

     % pgf90 -Minfo -fast -c ../pgi_mm.F

  2. Link the three MKL supplied libraries with the compiled object file. No -mp option links in the PGI serial run-time library. 

     % pgf90 -Minfo -o mm mm.o ../dclock_64.o -R/opt/mkl/lib/em64t -L/opt/mkl/lib/em64t 
   -lmkl_intel_lp64 -lmkl_sequential -lmkl_core

Determining which library to use for optimum performance involves a number of factors including array sizes, the choice of single or double precision arithmetic and the number of processors and/or cores available. Experimentation is the only sure way to know what's best for your situation.

Using MKL with PGI Visual Fortran
To use MKL with PGI Visual Fortran®, follow the directions for configuring MKL for Intel Visual Fortran in Section 4 of the MKL for Windows User's Guide. Substitute the appropriate mkl_pgi_thread* for the mkl_intel_thread* library as an Additional Dependency in the Configuration Properties > Linker > Input section of your project's Property Pages window. Enable the Process OpenMP Directives option through the Configuration Properties > Fortran > Language section as well to ensure the multi-treaded run-time layer is included.

See the Intel MKL documentation page for more information.

How can I use the linux LIBRARY_PATH and LDFLAGS environment with the PGI compilers?

The environment variable LIBRARY_PATH is used by some gcc drivers to add directories to the library search paths. This can be done with a few lines in the siterc file in the PGI install directory.

Create a siterc file with the following lines, or add the following lines to your existing siterc file; an alternative is to add these lines to a .mypgccrc or .mypgf90rc or .mypgf77rc or .mypgCCrc file in your home directory. 

# get the value of the environment variable LIBRARY_PATH
variable LIBRARY_PATH is environment(LD_LIBRARY_PATH);

# split this value at colons, separate by -L, prepend 1st one by -L
variable library_path is
default($if($LIBRARY_PATH,-L$replace($LIBRARY_PATH,":", -L)));

# add the -L arguments to the link line
append LDLIBARGS=$library_path;

Linker complains courbures.o(.text+0x2b2): undefined reference to `sgelsy_'

When linking LAPACK routines we recommend

-llapack -lblas -lpgftnrtl

However, in this case, the problem is sgelsy is not present in lapack version 2 -- it's a version 3 lapack routine, and we provide version 2. sgelsx can be called instead.

Linker complains ld: cannot open -lf2c: No such file or directory

This indicates that either g77 was not installed, and therefore, when linking in g77 compiled routines, the user's -g77libs flag results in a failure to find libf2c.a

It could also mean that g77 was updated to a version that uses libg2c.a instead of libf2c.a. This can usually be corrected by either installing g77 and reinstalling the pgi compiler, or by editing $PGI/PROD/bin/PRODrc and properly defining G77DIR and G77INC. (PROD == linux86 or solaris86 or nt86 or osf86).

Linker complains ld: cannot open -lgcc: No such file or directory

This means that the compiler cannot find the where gcc has been installed -- the linux system libraries are found in the gcc area.

This is usually the result of

  1. gcc has not been installed, or has been removed, or moved.
  2. gcc has been upgraded since the most recent installation.

The usual remedy is to reinstall the compilers, making sure that the gcc environment is set up properly ( compile and execute hello.c with gcc). This will ensure that our drivers point to the proper areas to find the system libraries.

To make sure things are correct, 

gcc -v hello.c

pgcc -v hello.c

Analyze the dialogue created and determine if the gcc libs being brought in are identical. (Not recommended) Modify $PGI/PROD/bin/PRODrc to make sure pointers are correct. (PROD == linux86 or nt86 or solaris86 or osf86).

gnu ld version 2.9.5 causes multiple ref errors with -mp and -Mconcur

The linker ld version 2.9.5 shows different behavior than the previous 2.9.1 version. As a result, programs linked with -Mconcur or -mp generate the following linker errors: 

/usr/pgi/linux86/lib/libpgc.a(init_dummy.o): In function `_mp_init2':
init_dummy.o(.text+0x10): multiple definition of `_mp_init2'
/usr/pgi/linux86/lib/libpgmp.a(init_pthr.o)(.text+0x200): first defined here
/usr/bin/ld: Warning: size of symbol `_mp_init2' changed from 149 to 17 in init_dummy.o
/usr/pgi/linux86/lib/libpgc.a(init_dummy.o)(.data+0x4): multiple definition of `_mp_tcpus'
/usr/pgi/linux86/lib/libpgmp.a(init_pthr.o)(.data+0xa8): first defined here
/usr/pgi/linux86/lib/libpgc.a(init_dummy.o): In function `_mp_term2':
init_dummy.o(.text+0x30): multiple definition of `_mp_term2'
/usr/pgi/linux86/lib/libpgmp.a(init_pthr.o)(.text+0x2e0): first defined here
/usr/bin/ld: Warning: size of symbol `_mp_term2' changed from 113 to 7 in init_dummy.o
/usr/pgi/linux86/lib/libpgc.a(init_dummy.o): In function `_mp_avlcpus':
init_dummy.o(.text+0x40): multiple definition of `_mp_avlcpus'
/usr/pgi/linux86/lib/libpgmp.a(init_pthr.o)(.text+0x360): first defined here
/usr/bin/ld: Warning: size of symbol `_mp_avlcpus' changed from 246 to 12 in init_dummy.o
(/usr/pgi/linux86/lib/libpgc.a)init_dummy2.o
/lib/libc.so.6 
(/usr/lib/libc_nonshared.a)fstat.oS
/usr/lib/gcc-lib/i486-suse-linux/2.95.2/crtend.o
/usr/lib/crtn.o
/usr/bin/ld: link errors found, deleting executable `linpkrd'
pgf77: /usr/bin/ld completed with exit code 1

We are also seeing the following errors 

/usr/pgi/linux86/lib/libpthread.so.0: undefined reference to `_mp_stackz'
/usr/pgi/linux86/lib/libpthread.so.0: undefined reference to `_mp_set_stkz'

The solution is to replace the file $PGI/linux86/lib/libpgmp.a with the file libpgmp.a located in the patch directory

ftp.pgroup.com/x86/3.1/linux86-patches/ld2.9.5

(Note: this file is also compatible with prior versions of ld).

PGHPF -- how do I direct the linker to a different mpi library?

You can override the path of libmpich.a by setting the environment variable HPF_MPI; that is, using csh syntax, 

setenv HPF_MPI /usr/local/packages/mpich-1.2.0/lib/libmpich.a

Another method is to update the file 

 $PGI/linux86/bin/linux86rc

to reflect the path of your version of libmpich.a. In linux86rc, you'll see a reference to the path 

 /usr/local/mpich/build/LINUX/ch_p4/lib/libmpich.a

Just replace this path with your path.

You'll also should recompile our 'hpf-mpi' interface file on your system to ensure that our hpf run-time will work with your the version of libmpich.a. If you obtain the file mpi.c from

ftp.pgroup.com/x86/3.1/linux86-patches/mpi/mpi.c

and update the archive 

 $PGI/linux86/lib/libpghpf_mpi.a

with the new mpi.o object file: 

cd $PGI/linux86/lib
cp libpghpf_mpi.a libpghpf.a-bak
ar crv libpghpf_mpi.a mpi.o

things should work.

How do I generate a link map from your compilers?

Memory maps are produced by your system's linker. Our compilers can pass switches directly to linkers by using '-Wl,' type switches.

For example, if -Map mapfilename creates a memory map in the file called mapfilename using the linux ld linker, then the compilers can create the same map file using 

pgf77 -o x x.f -Wl,-Map,mapfilename

The linker complains about missing symbols ___rouent2, ___rouret2, and ___linent2

When compiled programs are linked using the compilers, the following error messages occur. 

% pgf77  x.o
Linking:

/usr/pgi/linux86/lib/libpgc.a(rouret2.o)(.text+0x14): undefined reference to
`___rouent2'

/usr/pgi/linux86/lib/libpgc.a(rouret2.o)(.text+0x3f): undefined reference to
`___rouret2'

/usr/pgi/linux86/lib/libpgc.a(rouret2.o)(.text+0x74): undefined reference to
`___linent2'

The above occurs because the object file x.o was compiled with profile switches on, as in 

pgf77 -c -Mprof=func x.f

What the user needs to do is remember that the -Mprof switch causes the linker stage of compilation to bring in special profiler libraries which will satisfy the missing references. So the linking should be done with the profile switches set: 

pgf77 -Mprof=func x.o

Windows - when I try to create a DLL as in the User's Guide, I get 'cannot find -lgcc' message.

In order to create DLLs on a windows system, you need the entire cygwin environment. Our windows compilers come with a cygwin provided subset environment, but not the environment needed to create DLLs. To find our more about where/how to obtain the cygwin environment, go to www.cygwin.com. libgcc.a is part of the full cygwin environment.

Red Hat 8.0/release 4.0-2 - Changes to pthread use

The 11/08/02 download of 4.0-2 supports Red Hat 8.0. There is a change, however. libpgthread.so is not present, and the system lib libpthread.so is used instead. We created libpgthread.so because libpthread.so did not have large enough thread stack areas. But with RH 8.0, it appears libpthread.so will also get larger when you set the stacksize. 

limit stacksize unlimited

will increase the thread stack area as well as the regular stack.

The linker fails with undefined reference to `__ctype_b'

When compiled programs are linked using the compilers, the following error messages occur. 

% pgf77  x.o -lxyz
Linking:

/usr/pgi/linux86/lib/libxyz.a: undefined reference to `__ctype_b'

The above occurs because the library libxyz.a was built on a different linux version where a reference is made to __ctype_b instead of the more current __ctype_b_loc. The symbols __ctype_toupper and __ctype_tolower have a similar problem as well.

A workaround is to write a C routine that reconciles the symbols with the new ones.

ctype_wa.c 

#include <ctype.h>


__const unsigned short int *__ctype_b;
__const __int32_t *__ctype_tolower;
__const __int32_t *__ctype_toupper;


void ctSetup()
{
__ctype_b = *(__ctype_b_loc());
__ctype_toupper = *(__ctype_toupper_loc());
__ctype_tolower = *(__ctype_tolower_loc());
}



pgcc -c ctype_wa.c
pgf77 -c x.f ctype_wa.o

This is a linux problem, and this solution may not always work.

The linker complains about missing symbols __mth_i_aintx, __mth_i_rpowix, etc

When programs are compiled -fastsse, there is one or two libraries that need to be added to the link line. If you fail to put -fastsse on the link line, libpgsse1.a and libpgsse2.a will not be linked.

The solution is to add -fastsse to the link line as well.

The linker complains about unresolved external symbols when linking with -lacml on Windows.

In Windows, an error is reported when attempting to link in the ACML library. 

C:\tmp\matmul>  pgf77 -DBLAS -o matmul matmul.F -lacml -Munix
libacml.lib(acmlmallocc.obj) : warning LNK4217: locally defined symbol _malloc 
	imported in function _acmlmallocc_
libacml.lib(acmlgetenv.obj) : warning LNK4217: locally defined symbol _malloc 
	imported in function _acmlgetenv_
libacml.lib(acmlputenv.obj) : warning LNK4217: locally defined symbol _malloc im
ported in function _acmlputenv_
libacml.lib(acmlmallocc.obj) : warning LNK4217: locally defined symbol _free imp
orted in function _acmlfreec_
libacml.lib(acmlgetenv.obj) : warning LNK4049: locally defined symbol _free impo
rted
libacml.lib(acmlgetenv.obj) : warning LNK4217: locally defined symbol _getenv imported 
	in function _acmlgetenv_
libacml.lib(acmlgetenv.obj) : warning LNK4217: locally defined symbol _strncpy imported 
	in function _acmlgetenv_
libacml.lib(acmlputenv.obj) : warning LNK4217: locally defined symbol _strncpy imported 
	in function _acmlputenv_
libacml.lib(acmlputenv.obj) : error LNK2019: unresolved external symbol __imp__putenv 
	referenced in function _acmlputenv_
oldnames.lib(putenv.obi) : error LNK2001: unresolved external symbol __imp__putenv
libacml.lib(acmlputenv.obj) : error LNK2019: unresolved external symbol __imp__strncat 
	referenced in function _acmlputenv_
oldnames.lib(putenv.obi) : error LNK2001: unresolved external symbol __imp___putenv
matmul.exe : fatal error LNK1120: 3 unresolved externals

The ACML libraries shipped with PGI Windows products will resolve properly if linked using the -Bdynamic swtich. For example

C:\tmp\matmul>  pgf77 -DBLAS -o matmul matmul.F -Bdynamic -lacml -Munix

More information on using -Bdymanic can be found in Chapter 8 of the PGI User's Guide.

Click me