Request support/help for PBS with OpenMPI

cparrott
Posts: 13
Joined: Aug 13 2018

Re: Request support/help for PBS with OpenMPI

Post by cparrott » Mon Sep 09, 2019 10:49 am

FYI - I did a little more investigation here:

It seems Open MPI 4.0.1 has some bugs with regard to UCX support. I have not yet checked whether the recent Open MPI 4.0.2 release candidate has been fixed to work with UCX 1.6.0, or if you have to update to an Open MPI development snapshot from their git repo (which appears to be slated for release as Open MPI 4.1.0 eventually) to achieve compatibility with UCX 1.6.0.

The unfortunate thing is that documentation about which versions of UCX are compatible with which versions of Open MPI appears to be lacking, so a lot of this information only seems to be attainable through trial and error.

There do exist a few PGI compiler-related issues with Open MPI + UCX that have been opened in our bug tracker, and are currently under investigation. All this is to say that this is a very active area of development on multiple fronts right now - both on the Open MPI + UCX side and the PGI side - so stay tuned!

I am glad you found something to get you by in the meantime, and very sorry about the difficulties you have encountered. Will update once we have more progress to report.

Best regards,

+chris

sumseq
Posts: 133
Joined: Nov 27 2012

Re: Request support/help for PBS with OpenMPI

Post by sumseq » Tue Sep 10, 2019 9:33 am

Thanks for the update!

If you come across a working combo of versions (OpenMPI+UCX+PGI) that works for CUDA-aware MPI in OpenACC codes (especially on PBS) please let me know!

- Ron

Post Reply