PGI User Forum

[Jim Kirby]

Just getting the hang of this - I am using an Opteron cluster with 13 dual nodes.
I understand how to tell the codes how many processors to run on (NCPUS=n),
but I don't know how to control which processors are being used for those n threads.

Jim KIrby

[mkcolg]

Hi Jim,

I'm not 100% clear about what your asking but I think you might be confusing OpenMP with MPI programming. NCPUS is used with OpenMP to set the number of threads that are spawned on a stand-alone symetric multi-processor (SMP) system. OpenMP does not enable you to spawn threads on other nodes of your cluster. To run parallel executables across multiple cluster nodes, you'll need to use MPI instead.

Note that on some Linux distributions there are utilities, taskset and numactl, which do allow you to bind threads to a particular processor on a SMP system. MPI does not have this capabilbity.

- Mat

[Jim Kirby]

Mat - Thanks for the reply. I am talking about MPI or HPF - I know I can't
get beyond one node for the OpenMP or auto-parallelized versions.

I gather from your reply that any feature which would let me tie n threads to
n particular processors in the system is going to be in the form of a linux
system utility rather than a compiler option. I'll start looking in that direction now.

I posted a separate question in the installation section about not being able to
get the examples like matmul to run now - I just upgraded from 5.2 for workstation to CDK 6.0.

Jim Kirby

[mkcolg]

Hi Jim,

Just to be clear, taskset and numactl are for use on a stand-alone SMP system, not for use with MPI or HPF. With MPI you specify which n nodes to use with a 'machine' file. To use more than one processor on a given node, add the host to the machine file using 'hostname:n' where 'n' is the number of processors on the node. HPF uses the '-host' runtime flag to specify which nodes to use (Information about HPF runtime options can be found in Section 3 of the PGI HPF users guide).

Hope this helps,
Mat