PGI User Forum
 SearchSearch   MemberlistMemberlist     RegisterRegister   ProfileProfile    Log inLog in 

CUDA-x86.

libpgf90rtl undefined reference omp_set_schedule
Goto page 1, 2  Next
 
Post new topic   Reply to topic    PGI User Forum Forum Index -> Programming and Compiling
View previous topic :: View next topic  
Author Message
brian.eaton



Joined: 06 May 2011
Posts: 4

PostPosted: Sat Feb 08, 2014 1:33 pm    Post subject: libpgf90rtl undefined reference omp_set_schedule Reply with quote

I'm getting the following link failure while trying to build a simple
hello world program using the mpif90 for an openmpi installation built with
pgi-13.10 :

/usr/local/pgi/linux86/libso/libpgf90rtl.so: undefined reference to `omp_set_schedule'
/usr/local/pgi/linux86/libso/libpgf90rtl.so: undefined reference to `omp_get_schedule'
pgf90-Fatal-linker completed with exit code 1

Details of compiler version:

% pgf90 --version
pgf90 13.10-0 64-bit target on x86-64 Linux -tp penryn
The Portland Group - PGI Compilers and Tools
Copyright (c) 2013, NVIDIA CORPORATION. All rights reserved.

The pgf90 command issued by mpif90 is:

% mpif90 --showme
pgf90 -I/usr/mpi/pgi/openmpi-1.4.3-qlc/include
-I/usr/mpi/pgi/openmpi-1.4.3-qlc/lib64
-L/usr/mpi/pgi/openmpi-1.4.3-qlc/lib64
-lmpi_f90 -lmpi_f77 -lmpi -lopen-rte -lopen-pal
-ldl -Wl,--export-dynamic -lnsl -lutil -lpthread -ldl

I've issued the pgf90 command directly, and supplemented it with
-Wl,-M -Wl,--cref to obtain load map with cross references. What
I find is that libpgf90rtl is not supplying any references for the
hello world executable, but is causing the link to fail because of
the undefined reference problem.

I've found a workaround which is to add -Wl,--allow-shlib-undefined to
the load flags. But it doesn't seem like I should need to do this.

Are these undefined references in libpgf90rtl needed for something?
They don't appear in version 11.5 of the library which I was using
before updating to 13.10.
Back to top
View user's profile
mkcolg



Joined: 30 Jun 2004
Posts: 6129
Location: The Portland Group Inc.

PostPosted: Tue Feb 11, 2014 5:07 pm    Post subject: Reply with quote

Hi Brian,

It looks like for some reason the our mp library isn't getting added to the link. This can occur if the flag "-nomp" is added, but I don't see it in your case.

One thing I don't understand is why the path listed is for the 32-bit lib directory (/usr/local/pgi/linux86/libso). However, "libpgf90rtl.so" is only available in 64-bits and everything else indicates that you're doing a 64-bit compile. Is the directory name a typo? or did things get moved around in your installation?

If things were moved, can you check if "libpgmp.so" was moved as well? Maybe try adding "-L/usr/local/pgi/linux86/libso -lpgmp"?

- Mat
Back to top
View user's profile
brian.eaton



Joined: 06 May 2011
Posts: 4

PostPosted: Fri Feb 14, 2014 3:57 pm    Post subject: Reply with quote

Hi Mat,

Actually the pgmp lib is being added to the link command. I should have
included this in my original post. Using the -v option to pgf90 shows that
the actual link command resulting from the pgf90 command in the original
post is:

/usr/bin/ld /usr/lib64/crt1.o /usr/lib64/crti.o
/usr/local/pgi-pgcc-pghf-13.10/linux86-64/13.10/lib/trace_init.o
/usr/lib/gcc/x86_64-redhat-linux/4.4.7/crtbegin.o
/usr/local/pgi-pgcc-pghf-13.10/linux86-64/13.10/lib/initmp.o
/usr/local/pgi-pgcc-pghf-13.10/linux86-64/13.10/lib/f90main.o -m elf_x86_64
-dynamic-linker /lib64/ld-linux-x86-64.so.2
/usr/local/pgi-pgcc-pghf-13.10/linux86-64/13.10/lib/pgi.ld
-L/usr/mpi/pgi/openmpi-1.4.3-qlc/lib64
-L/usr/local/pgi-pgcc-pghf-13.10/linux86-64/13.10/lib -L/usr/lib64
-L/usr/lib/gcc/x86_64-redhat-linux/4.4.7 -lmpi_f90 -lmpi_f77 -lmpi
-lopen-rte -lopen-pal --export-dynamic -M --cref -t --allow-shlib-undefined
/tmp/pgf90zsOblXiHsi33.o -rpath
/usr/local/pgi-pgcc-pghf-13.10/linux86-64/13.10/lib -laccapid -lpgmp -lnuma
-lpthread -lpgf90 -lpgf90_rpm1 -lpgf902 -lpgf90rtl -lpgftnrtl -lpgmp
-lnspgc -lpgc -lrt -lpthread -lm -lgcc -lc -lgcc
/usr/lib/gcc/x86_64-redhat-linux/4.4.7/crtend.o /usr/lib64/crtn.o

The soft links are resolved in the above output, so you can see that in our
installation linux86 is a link to linux86-64.

Also, in our local installation /usr/local/pgi/linux86/libso resolves to
/usr/local/pgi-pgcc-pghf-13.10/linux86-64/13.10/libso

The above link command also includes the libs that were added by the mpif90
invocation of pgf90.

However, while the link line contains
/usr/local/pgi-pgcc-pghf-13.10/linux86-64/13.10/lib
it does *not* contain
/usr/local/pgi-pgcc-pghf-13.10/linux86-64/13.10/libso


If I add "-L/usr/local/pgi/linux86/libso" to the pgf90 command as you
suggest (and remove -Wl,--allow-shlib-undefined) then the build of the
hello world test program works. But without explicitly adding it the build
fails.

Thanks for the suggestion to add the libso path to the link command. That
gives me a workaround which I like better than my original. But I'm still
trying to figure out whether this is the best solution. I have a suspicion
that something could be done in the openmpi installation so that it
wouldn't be necessary for the user to add this argument to the mpif90
commandline.


Note that /usr/local/pgi/linux86/libso is in LD_LIBRARY_PATH.


I guess my questions are the following:

. Should I need to manually add "-L/usr/local/pgi/linux86/libso" to my link
command. Shouldn't ld know to look in LD_LIBRARY_PATH which does contain
the libso path?

. Why does pgf90 only add the pgi/linux86-64/lib path and not
pgi/linux86-64/libso? Is there an option I should be using that tells
pgf90 to add the libso path?

. There is a pgmp lib in pgi/linux86-64/lib. It appears to have
definitions for the omp_set_schedule and omp_get_schedule references
which are causing the link failure. Why isn't libpgmp.a satisfying those
references?

Thanks,
Brian
Back to top
View user's profile
mkcolg



Joined: 30 Jun 2004
Posts: 6129
Location: The Portland Group Inc.

PostPosted: Fri Feb 14, 2014 6:06 pm    Post subject: Reply with quote

Hi Brian,

This one has got me confused!

What I don't understand is why "libso/libpgf90rtl.so" is getting linked in the first place. Nowhere on your link line does "libso" show up. Maybe it's getting picked up via LD_LIBRARY_PATH, but I'd think the rpath would take precedence.

Quote:
. Should I need to manually add "-L/usr/local/pgi/linux86/libso" to my link
command. Shouldn't ld know to look in LD_LIBRARY_PATH which does contain the libso path?
No, you shouldn't. And if LD_LIBRARY_PATH is being used to find "libpgf90rtl.so", why isn't it finding "libpgmp.so"? odd.

Quote:
. Why does pgf90 only add the pgi/linux86-64/lib path and not
pgi/linux86-64/libso? Is there an option I should be using that tells
pgf90 to add the libso path?
The default is to link using the static libraries. Adding "-Bdynamic" or "-fpic" will cause the driver to use "libso" instead of "lib".

Quote:
. There is a pgmp lib in pgi/linux86-64/lib. It appears to have
definitions for the omp_set_schedule and omp_get_schedule references
which are causing the link failure. Why isn't libpgmp.a satisfying those
references?
While I'm not 100% positive, I don't think a shared object can resolve it's symbols to a static library.

The one thought that comes to mind is if someone has put links to the shared objects in your lib directory but missed "libpgmp.so"? That might cause this.

If not, then can you add the flag "-Wl,--verbose" to have the linker show us how it's resolving the libraries?

- Mat
Back to top
View user's profile
brian.eaton



Joined: 06 May 2011
Posts: 4

PostPosted: Sat Feb 15, 2014 2:47 pm    Post subject: Reply with quote

Hi Mat,

I've run the following command:

Code:
 mpif90 -Wl,-M,--cref,--verbose test.f90 >| mpif90-test.out 2>&1

where test.f90 contains the 3 line hello world program.

The --verbose output contains:

Quote:
libpgf90rtl.so needed by /usr/mpi/pgi/openmpi-1.4.3-qlc/lib64/libmpi_f90.so
found libpgf90rtl.so at /usr/local/pgi/linux86/libso/libpgf90rtl.so

So it is the openmpi libs added by the mpif90 command which is triggering
the problem, as expected.

But the --verbose output indicates the linker was able to locate
libpgf90rtl.so (I assume via the LD_LIBRARY_PATH), and yet doesn't seem
able to figure out that omp_set_schedule is resolved by libpgmp.so in the
same directory.

So this is still feeling like something strange in the pgi shared libs.

I didn't see a way to attach the output which contains all the linkage
info, so I put it here: ftp://ftp.cgd.ucar.edu/pub/eaton/mpif90-test.out.gz

Thanks for your help with this!
Brian
Back to top
View user's profile
Display posts from previous:   
Post new topic   Reply to topic    PGI User Forum Forum Index -> Programming and Compiling All times are GMT - 7 Hours
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © phpBB Group