PGI User Forum
 SearchSearch   MemberlistMemberlist     RegisterRegister   ProfileProfile    Log inLog in 

CUDA-x86.

pgi 11.8 and openmpi 1.4.3
Goto page Previous  1, 2, 3
 
Post new topic   Reply to topic    PGI User Forum Forum Index -> Programming and Compiling
View previous topic :: View next topic  
Author Message
mkcolg



Joined: 30 Jun 2004
Posts: 6129
Location: The Portland Group Inc.

PostPosted: Tue Nov 22, 2011 10:06 am    Post subject: Reply with quote

Hi Francesco,

I'm wondering which libnuma.so library the OpenMPI runtime is loading. Can you look in /usr/lib to see if libnuma is there? and if so, which version? If it's not there, openMPI may be picking up the dummy libnuma that we install on non-NUMA systems.

- Mat
Back to top
View user's profile
franzisko



Joined: 11 Jan 2011
Posts: 25

PostPosted: Tue Nov 22, 2011 12:03 pm    Post subject: Reply with quote

Hi Mat,

there is libnuma.so pointing to libnuma.so.1 in /usr/lib and /usr/lib64. When configuring Openmpi I specify --with-libnuma=/usr/. Changing this directory configure script fails.

Typing
Code:
rpm -qf /usr/lib64/libnuma.so.1

I get
Code:
numactl-0.9.8-12.el5_6.x86_64

Typing
Code:
yum info numactl.x86_64

I get
Code:
Name       : numactl
Arch       : x86_64
Version    : 0.9.8
Release    : 12.el5_6
Size       : 96 k
Repo       : installed
Summary    : libreria per migliorare le prestazioni delle macchine con Non Uniform Memory Access.
URL        : ftp://ftp.suse.com/pub/people/ak/numa/
License    : LGPL/GPL
Description: Supporto per policy Simple NUMA. Consiste in un programma numactl che esegue
           : altri programmi con una specifica policy NUMA. Contiene libnuma, utilizzata per le
           : allocazioni nelle applicazioni attraverso una policy NUMA.

we are very thankful to you for all your help

Francesco
Back to top
View user's profile
franzisko



Joined: 11 Jan 2011
Posts: 25

PostPosted: Mon Nov 28, 2011 10:03 am    Post subject: Reply with quote

Hi Mat,

to use maffinity avoiding segfault, I found it is possibile to select a policy different from libnuma that is first_use. I do not know any performance issue but it seems a good solution to avoid segfault with maffinity.

mpirun --mca maffinity first_use <executable>

works fine for us. I inserted the option in the global openmpi mca configuration file so I do not have to specify every time this option.

thanks
Francesco
Back to top
View user's profile
David Gunter



Joined: 15 Sep 2005
Posts: 25

PostPosted: Tue Apr 24, 2012 7:52 am    Post subject: Reply with quote

We are hitting this exact problem with PGI 11.10, 12.1 and 12.4. We can either build a poor-performing OMPI or one that refuses to work at all.

At this point we've reached the point where we can no longer offer PGI for our customer base and will most likely not renew our licenses this year.
Back to top
View user's profile
Display posts from previous:   
Post new topic   Reply to topic    PGI User Forum Forum Index -> Programming and Compiling All times are GMT - 7 Hours
Goto page Previous  1, 2, 3
Page 3 of 3

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © phpBB Group