PGI User Forum
 SearchSearch   MemberlistMemberlist     RegisterRegister   ProfileProfile    Log inLog in 

CUDA-x86.

NaN's in code that worked with Intel compiler

 
Post new topic   Reply to topic    PGI User Forum Forum Index -> Debugging and Profiling
View previous topic :: View next topic  
Author Message
Jon Slavin



Joined: 29 Mar 2011
Posts: 2

PostPosted: Tue Mar 29, 2011 11:56 am    Post subject: NaN's in code that worked with Intel compiler Reply with quote

To all:

I'm new to using pgfortran. Up to recently I've been using the Intel fortran compiler, but am porting the code to run on our cluster which doesn't have that compiler. The code I'm compiling has been working perfectly on a very similar system (Linux CentOS 64 bit), but when I compile with pgfortran and run it, I get NaN's.

I found a posting about detecting NaN's and have used the code listed to find when a certain variable is NaN. The oddest thing is that when I print the values of two other vairables that are summed to give the value of the variable, each of them are ordinary reasonable numbers. That makes me think that there must be some sort of memory error. Any advice on what could be the cause of this and how to diagnose the problem would be appreciated. As far as I can tell pgfortran doesn't offer much in the way of runtime checking, but if I'm mistaken about that, I'd like to know how to enable it.

Thanks,
Jon
Back to top
View user's profile
mkcolg



Joined: 30 Jun 2004
Posts: 5815
Location: The Portland Group Inc.

PostPosted: Tue Mar 29, 2011 1:15 pm    Post subject: Reply with quote

Hi Jon,

The flag "-Ktrap" will detect at runtime IEEE trap conditions such as divide by zero, underflow, overflow, etc, that may be the cause of the NaN. (Please see 'pgfortran -help -Ktrap" for the sub-options or the PGI User's Guide).

For memory issues, I usually recommend the Valgrind utility (www.valgrind.org).

What options are you using for both the PGI and Intel builds? Does the error occur at low optimization (-g)? Is your application multi-threaded (OpenMP or MPI)?

- Mat
Back to top
View user's profile
Jon Slavin



Joined: 29 Mar 2011
Posts: 2

PostPosted: Wed Mar 30, 2011 6:53 am    Post subject: Reply with quote

Hi Mat,

Thanks for the reply. I'll try using -Ktrap and may request that valgrind be installed on the cluster.

I have been using -C -g when compiling with pgfortran (but optimized when compiling with Intel ifort). The code does use MPI, though I created a non-MPI version for debugging purposes and am still getting the NaN's.

I should mention that the code is mixed F77 and F90 (i.e. some subroutines/functions are F77 others are F90) in case that makes any difference.

Thanks,
Jon
Back to top
View user's profile
mkcolg



Joined: 30 Jun 2004
Posts: 5815
Location: The Portland Group Inc.

PostPosted: Wed Mar 30, 2011 9:06 am    Post subject: Reply with quote

Hi Jon,

Since the problem occurs with "-g", it's more likely a problem in the code.

One culprit may be an uninitialized variable. At low optimization, variables are loaded and stored from memory while at higher optimization, values are more likely to be stored in registers. If the variable happens to be stored in a register, it may have a consistent valid initial value, while if loaded from memory may contain garbage. Valgrind will be able to find uninitialized memory reads (UMR).

Other causes may be that that an optimization reorders the code so the program error is masked. In this case, try compiling at full optimization and see if the problem goes away. If it does, then uses a binary search method where you compile half the source file with optimization and the other half without. Continue until you find exactly which file causes the NaN. From there, you can use the "opt" directive to selectively optimize each routine in the file. Continue until you've narrowed it down the exact routine. Finally, run the code in the debugger (PGDBG) and step through the routine to determine the exact cause of the NaN.

Hope this helps,
Mat
Back to top
View user's profile
Display posts from previous:   
Post new topic   Reply to topic    PGI User Forum Forum Index -> Debugging and Profiling All times are GMT - 7 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © phpBB Group