PGI User Forum
 SearchSearch   MemberlistMemberlist     RegisterRegister   ProfileProfile    Log inLog in 

CUDA-x86.

-Mprof=mpich2 creates pgprof.out* for C but not for F90

 
Post new topic   Reply to topic    PGI User Forum Forum Index -> Programming and Compiling
View previous topic :: View next topic  
Author Message
rikisyo



Joined: 20 Jun 2013
Posts: 7

PostPosted: Fri Aug 02, 2013 11:27 am    Post subject: -Mprof=mpich2 creates pgprof.out* for C but not for F90 Reply with quote

I am trying to do MPI profiling with PGI compilers and profiler. As directed by the documentation I use add -Mprof=mpich2,time when compiling my program, but the executable only generates pgprof.out, but not other pgprof.out* files. I can't profile my MPI programs without these per-process profiling data.

To investigate the problem I use a small "hello world" with minimal MPI calls, written in both C and F90. To my surprise the C version always generates correct pgprof.out* for all processes, but the F90 version always fails.

My C code:
Code:

#include <stdio.h>
#include <mpi.h>

int main (argc, argv)
     int argc;
     char *argv[];
{
  int rank, size;

  MPI_Init (&argc, &argv);      /* starts MPI */
  MPI_Comm_rank (MPI_COMM_WORLD, &rank);        /* get current process id */
  MPI_Comm_size (MPI_COMM_WORLD, &size);        /* get number of processes */
  printf( "Hello world from process %d of %d\n", rank, size );
  MPI_Finalize();
  return 0;
}

Compile command and output:
Code:

$ mpicc -Mprof=time,mpich2 -o hellompi hellompi.c -v
mpicc for MPICH2 version 1.4.1p1

/usr/local/pgi/linux86-64/12.5/bin/pgc hellompi.c -opt 1 -x 119 0xa10000 -x 122 0x40 -x 123 0x1000 -x 127 4 -x 127 17 -x 19 0x400000 -x 28 0x40000 -x 120 0x10000000 -x 70 0x8000 -x 122 1 -x 125 0x20000 -quad -x 59 4 -x 59 4 -tp core2 -x 120 0x1000 -astype 0 -stdinc /usr/local/pgi/linux86-64/12.5/include:/usr/local/include:/usr/lib/gcc/x86_64-redhat-linux/3.4.6/include:/usr/lib/gcc/x86_64-redhat-linux/3.4.6/include:/usr/include -def unix -def __unix -def __unix__ -def linux -def __linux -def __linux__ -def __NO_MATH_INLINES -def __x86_64 -def __x86_64__ -def __LONG_MAX__=9223372036854775807L -def '__SIZE_TYPE__=unsigned long int' -def '__PTRDIFF_TYPE__=long int' -def __THROW= -def __extension__= -def __amd_64__amd64__ -def __k8 -def __k8__ -def __SSE__ -def __MMX__ -def __SSE2__ -def __SSE3__ -def __SSSE3__ -predicate '#machine(x86_64) #lint(off) #system(posix) #cpu(x86_64)' -idir /usr/local/pgi/linux86-64/2012/mpi2/mpich/include -idir /usr/local/apps/mpich2/pgi105x64/1.4.1p1/include -cmdline '+pgcc hellompi.c -pgf90libs -Mprof=time,mpich2 -o hellompi -v -I/usr/local/apps/mpich2/pgi105x64/1.4.1p1/include -L/usr/local/apps/mpich2/pgi105x64/1.4.1p1/lib -Wl,-rpath,/usr/local/apps/mpich2/pgi105x64/1.4.1p1/lib -lmpich -lopa -lmpl -lrt -lpthread' -x 123 0x80000000 -x 123 4 -x 2 0x400 -x 119 0x20 -def __pgnu_vsn=30406 -alwaysinline /usr/local/pgi/linux86-64/12.5/lib/libintrinsics.il 4 -x 120 0x200000 -x 120 0x1000 -profile lines -x 2 0x100000 -ccff /tmp/pgccIEcdMPOVwMPk.ccff -asm /tmp/pgccsEcd2s2FWIYV.s
PGC-I-0222-Redundant definition for symbol __THROW (/usr/include/sys/cdefs.h: 63)
PGC-I-0222-Redundant definition for symbol __extension__ (/usr/include/sys/cdefs.h: 287)
PGC/x86-64 Linux 12.5-0: compilation completed with informational messages

/usr/bin/as /tmp/pgccsEcd2s2FWIYV.s -I/usr/local/pgi/linux86-64/2012/mpi2/mpich/include/ -I/usr/local/apps/mpich2/pgi105x64/1.4.1p1/include/ -o /tmp/pgccYEcdw2Tfr2zu.o

/usr/local/pgi/linux86-64/12.5/bin/pgappend -noerror /tmp/pgccYEcdw2Tfr2zu.o -name .CCFF /tmp/pgccIEcdMPOVwMPk.ccff -cz -keep

/usr/bin/ld /usr/lib64/crt1.o /usr/lib64/crti.o /usr/local/pgi/linux86-64/12.5/lib/trace_init.o /usr/lib/gcc/x86_64-redhat-linux/3.4.6/crtbegin.o /usr/local/pgi/linux86-64/12.5/lib/initmp.o /usr/local/pgi/linux86-64/12.5/lib/init_pgpf.o -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 /usr/local/pgi/linux86-64/12.5/lib/pgi.ld -L/usr/local/apps/mpich2/pgi105x64/1.4.1p1/lib -L/usr/local/pgi/linux86-64/12.5/lib -L/usr/lib64 -L/usr/lib/gcc/x86_64-redhat-linux/3.4.6 /tmp/pgccYEcdw2Tfr2zu.o -rpath /usr/local/apps/mpich2/pgi105x64/1.4.1p1/lib -lmpich -lopa -lmpl -lrt -lpthread -rpath /usr/local/pgi/linux86-64/2012/mpi2/mpich/lib -rpath /usr/local/pgi/linux86-64/2012/mpi2/mpich/lib -rpath /usr/local/pgi/linux86-64/12.5/lib -o hellompi -L/usr/local/pgi/linux86-64/2012/mpi2/mpich/lib -L/usr/local/pgi/linux86-64/2012/mpi2/mpich/lib -lfmpich -lpgnod_prof_time -lpgnod_prof_mpi2 -lmpich -lopa -lmpl -lpthread /usr/local/pgi/linux86-64/12.5/lib/nonuma.o -lpgmp -lpthread -lpgf90 -lpgf90_rpm1 -lpgf902 -lpgf90rtl -lpgftnrtl -lnspgc -lpgc -lrt -lpthread -lm -lgcc -lc -lgcc /usr/lib/gcc/x86_64-redhat-linux/3.4.6/crtend.o /usr/lib64/crtn.o
Unlinking /tmp/pgccsEcd2s2FWIYV.s
Unlinking /tmp/pgccIEcdMPOVwMPk.ccff
Unlinking /tmp/pgccYEcdw2Tfr2zu.o


My F90 code:
Code:

program hello

include "mpif.h"

integer rank, size, ierror, tag, status(MPI_STATUS_SIZE)

call PMPI_INIT(ierror)
call PMPI_COMM_SIZE(MPI_COMM_WORLD, size, ierror)
call PMPI_COMM_RANK(MPI_COMM_WORLD, rank, ierror)
print*, 'node', rank, ': Hello world'
call PMPI_FINALIZE(ierror)

end program


Compile command and output:
Code:

$ mpif90 -Mprof=time,mpich2 -o hellompi hellompi.f90 -v                                                                                                                         [2/2390]
mpif90 for MPICH2 version 1.4.1p1

/usr/local/pgi/linux86-64/12.5/bin/pgf901 hellompi.f90 -opt 1 -nohpf -nostatic -x 19 0x400000 -quad -x 59 4 -x 59 4 -x 15 2 -x 49 0x400004 -x 51 0x20 -x 57 0x4c -x 58 0x10000 -x 124 0x1000 -x 57 0xfb0000 -x 58
0x78031040 -x 48 4608 -x 49 0x100 -x 120 0x200 -stdinc /usr/local/pgi/linux86-64/12.5/include:/usr/local/include:/usr/lib/gcc/x86_64-redhat-linux/3.4.6/include:/usr/lib/gcc/x86_64-redhat-linux/3.4.6/include:/us
r/include -def unix -def __unix -def __unix__ -def linux -def __linux -def __linux__ -def __NO_MATH_INLINES -def __x86_64 -def __x86_64__ -def __LONG_MAX__=9223372036854775807L -def '__SIZE_TYPE__=unsigned long
 int' -def '__PTRDIFF_TYPE__=long int' -def __THROW= -def __extension__= -def __amd_64__amd64__ -def __k8 -def __k8__ -def __SSE__ -def __MMX__ -def __SSE2__ -def __SSE3__ -def __SSSE3__ -idir /usr/local/pgi/li
nux86-64/2012/mpi2/mpich/include -idir /usr/local/apps/mpich2/pgi105x64/1.4.1p1/include -idir /usr/local/apps/mpich2/pgi105x64/1.4.1p1/include -freeform -vect 48 -y 54 1 -modexport /tmp/pgfortranCsgduPH_g1tc.cm
od -modindex /tmp/pgfortran8sgd0ZUWTv7D.cmdx -output /tmp/pgfortran8sgd0sNtX2c1.ilm
PGF90-I-0035-Predefined intrinsic size loses intrinsic property (hellompi.f90: 8)
  1 inform,   0 warnings,   0 severes, 0 fatal for hello
PGF90/x86-64 Linux 12.5-0: compilation completed with informational messages

/usr/local/pgi/linux86-64/12.5/bin/pgf902 /tmp/pgfortran8sgd0sNtX2c1.ilm -fn hellompi.f90 -opt 1 -x 51 0x20 -x 119 0xa10000 -x 122 0x40 -x 123 0x1000 -x 127 4 -x 127 17 -x 19 0x400000 -x 28 0x40000 -x 120 0x100
00000 -x 70 0x8000 -x 122 1 -x 125 0x20000 -quad -x 59 4 -x 59 4 -tp core2 -x 120 0x1000 -x 124 0x1400 -y 15 2 -x 57 0x3b0000 -x 58 0x48000000 -x 49 0x100 -x 120 0x200 -astype 0 -x 124 1 -x 120 0x1000 -profile
lines -x 2 0x100000 -cmdline '+pgfortran hellompi.f90 -Mprof=time,mpich2 -o hellompi -v -I/usr/local/apps/mpich2/pgi105x64/1.4.1p1/include -I/usr/local/apps/mpich2/pgi105x64/1.4.1p1/include -L/usr/local/apps/mp
ich2/pgi105x64/1.4.1p1/lib -lmpichf90 -Wl,-rpath -Wl,/usr/local/apps/mpich2/pgi105x64/1.4.1p1/lib -lmpichf90 -lmpich -lopa -lmpl -lrt -lpthread' -ccff /tmp/pgfortran8sgd0KpNk3zq.ccff -asm /tmp/pgfortranCsgdusLK
LVya.s
  0 inform,   0 warnings,   0 severes, 0 fatal for hello
PGF90/x86-64 Linux 12.5-0: compilation successful

/usr/bin/as /tmp/pgfortranCsgdusLKLVya.s -I/usr/local/pgi/linux86-64/2012/mpi2/mpich/include/ -I/usr/local/apps/mpich2/pgi105x64/1.4.1p1/include/ -I/usr/local/apps/mpich2/pgi105x64/1.4.1p1/include/ -o /tmp/pgfo
rtranCsgdulmVl6tq.o

/usr/local/pgi/linux86-64/12.5/bin/pgappend -noerror /tmp/pgfortranCsgdulmVl6tq.o -name .CCFF /tmp/pgfortran8sgd0KpNk3zq.ccff -cz -keep -name .IPDINFO /tmp/pgfortranCsgduPH_g1tc.cmod -name .IPEINFO /tmp/pgfortr
an8sgd0ZUWTv7D.cmdx

/usr/bin/ld /usr/lib64/crt1.o /usr/lib64/crti.o /usr/local/pgi/linux86-64/12.5/lib/trace_init.o /usr/lib/gcc/x86_64-redhat-linux/3.4.6/crtbegin.o /usr/local/pgi/linux86-64/12.5/lib/initmp.o /usr/local/pgi/linux
86-64/12.5/lib/init_pgpf.o /usr/local/pgi/linux86-64/12.5/lib/f90main.o -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 /usr/local/pgi/linux86-64/12.5/lib/pgi.ld -L/usr/local/apps/mpich2/pgi105x64/1.4
.1p1/lib -L/usr/local/pgi/linux86-64/12.5/lib -L/usr/lib64 -L/usr/lib/gcc/x86_64-redhat-linux/3.4.6 /tmp/pgfortranCsgdulmVl6tq.o -lmpichf90 -rpath /usr/local/apps/mpich2/pgi105x64/1.4.1p1/lib -lmpichf90 -lmpich
 -lopa -lmpl -lrt -lpthread -rpath /usr/local/pgi/linux86-64/2012/mpi2/mpich/lib -rpath /usr/local/pgi/linux86-64/2012/mpi2/mpich/lib -rpath /usr/local/pgi/linux86-64/12.5/lib -o hellompi -L/usr/local/pgi/linux
86-64/2012/mpi2/mpich/lib -L/usr/local/pgi/linux86-64/2012/mpi2/mpich/lib -lfmpich -lmpichf90 -lpgnod_prof_time -lpgnod_prof_mpi2 -lmpich -lopa -lmpl -lpthread /usr/local/pgi/linux86-64/12.5/lib/nonuma.o -lpgmp
 -lpthread -lpgf90 -lpgf90_rpm1 -lpgf902 -lpgf90rtl -lpgftnrtl -lnspgc -lpgc -lrt -lpthread -lm -lgcc -lc -lgcc /usr/lib/gcc/x86_64-redhat-linux/3.4.6/crtend.o /usr/lib64/crtn.o
Unlinking /tmp/pgfortran8sgd0sNtX2c1.ilm
Unlinking /tmp/pgfortranCsgduPH_g1tc.cmod
Unlinking /tmp/pgfortran8sgd0ZUWTv7D.cmdx
Unlinking /tmp/pgfortranCsgdusLKLVya.s
Unlinking /tmp/pgfortran8sgd0KpNk3zq.ccff
Unlinking /tmp/pgfortranCsgdulmVl6tq.o


I launch both executables with
Code:

mpiexec -np 4 ./hellompi
Back to top
View user's profile
mkcolg



Joined: 30 Jun 2004
Posts: 6124
Location: The Portland Group Inc.

PostPosted: Mon Aug 05, 2013 2:45 pm    Post subject: Reply with quote

Hi rikisyo,

I sent this over to one of our Tools Engineers but unfortunately he couldn't recreate your error using either PGI 12.5 or 13.6 so don't know what's wrong.

One thing you could try would be to use ‘pgf90’ instead of ‘mpif90’. You can set MPIDIR to get the driver to pick up the MPICH-2 libraries. We don’t see anything wrong with the linker output in your test case but it’s probably worth a try.

- Mat
Back to top
View user's profile
rikisyo



Joined: 20 Jun 2013
Posts: 7

PostPosted: Wed Aug 07, 2013 7:52 am    Post subject: Reply with quote

mkcolg wrote:
Hi rikisyo,

I sent this over to one of our Tools Engineers but unfortunately he couldn't recreate your error using either PGI 12.5 or 13.6 so don't know what's wrong.

One thing you could try would be to use ‘pgf90’ instead of ‘mpif90’. You can set MPIDIR to get the driver to pick up the MPICH-2 libraries. We don’t see anything wrong with the linker output in your test case but it’s probably worth a try.

- Mat


I tried the suggestions but all failed. Thanks anyway!
Back to top
View user's profile
Display posts from previous:   
Post new topic   Reply to topic    PGI User Forum Forum Index -> Programming and Compiling All times are GMT - 7 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © phpBB Group