PGI User Forum
 SearchSearch   MemberlistMemberlist     RegisterRegister   ProfileProfile    Log inLog in 

CUDA-x86.

pgdbg problem for large Fortran program

 
Post new topic   Reply to topic    PGI User Forum Forum Index -> Debugging and Profiling
View previous topic :: View next topic  
Author Message
glendening



Joined: 21 Dec 2004
Posts: 9

PostPosted: Mon Feb 27, 2006 2:17 pm    Post subject: pgdbg problem for large Fortran program Reply with quote

Previously I've only used pgdbg v6.0-2 for relatively small "toy"
Fortran programs, for which it has worked satisfactorily. But I'm now
trying to use it for a much larger Fortran program, the WRF ndown
program which when compiled with -g has an executable size of ~52MB.
When I do so I find that breakpoints can be set only at a small
fraction of the executable statements, as illustrated by the GUI
snapshot at
http://www.drjack.info/MISC/pgdbg_ndown_snapshot.png
where I have set all possible breakpoints. Obviously this makes
debugging nearly impossible, since critical sections cannot be
examined on a step-by-step basis. And I can't set a breakpoint at any
subroutine call.

The same problem occurs when using the "-text" option, so it can't be
a GUI problem. I also note that pgdbg takes ~10 minutes to finish
loading for both cases. But other than having limited breakpoints,
pgbdg does seem to work i.e. I can see no obvious errors when I do
stop at an available breakpoint (though I have not made extensive
tests.)

Is this a known limitation/problem of pgdbg?? Do I need more RAM?? I
tried running pgdbg on several other WRF executables of similar size
and had the same problem with all, so it is not just a problem with a
single executable. (I'm running RHEL3 on a dual Athlon with 2GB RAM
and exited all other programs to free RAM.)
Back to top
View user's profile
mkcolg



Joined: 30 Jun 2004
Posts: 6134
Location: The Portland Group Inc.

PostPosted: Mon Feb 27, 2006 3:48 pm    Post subject: Reply with quote

Hi Jack,

You should be able to set a break point at each basic block. However, if you've added any optimization with "-g", then the blocks may be moved around and changed. At least from your screen shot, this appears to be what might be happening. Just ot confirm, did you compile with just "-g" or with "-g" plus an optimization? Note that it's fine to debug optimized code, but you may need to mix the program source with assembly and use "nexti" or "stepi" to step through each assembly instruction. To mix source code with the generated assembly, select "Mixed" from the pull-down at the bottom right of the debugger interface.

As for the load time issue. I've complained about this one to, but our tools group insist that there is nothing they can do. It simply takes a long time to read in all the dwarf information from very large programs. I have found that the best way to get around this problem is to compile only portions of your code with "-g" and the rest with "-O0". I'm currently debugging a very large C++ code, and this techinque has worked well.

- Mat
Back to top
View user's profile
glendening



Joined: 21 Dec 2004
Posts: 9

PostPosted: Tue Feb 28, 2006 1:20 pm    Post subject: Reply with quote

Mat,

That was it, I had been including -O2 optimization and changing to -O0
does let me set breakpoints at all lines. I've been used to a
debugger which automatically turns off optimization with -g so was not
acquainted with PGI's allowing debugging of optimized code (which I
have since read up on). That should be an advantage in this case
since the problem code bombs using -O2 but not with -O1 - I assume I
can compile using -O2 and then use the available breakpoints to at
least find the block of code within which the problem lies. I will then try
"mixed" technique you mention to hopefully find the exact problem line.

Thanks for your help as I completely misread where the problem lay.

Jack
Back to top
View user's profile
mkcolg



Joined: 30 Jun 2004
Posts: 6134
Location: The Portland Group Inc.

PostPosted: Tue Feb 28, 2006 5:14 pm    Post subject: Reply with quote

Hi Jack,

When you say the code 'bombs' do you mean the code gets a segmentation violation, the program gives an internal error, or are your getting wrong answers? If your getting a seg fault, then the debugger should be able to point you to the exact spot where it's occuring. If your program has it's own error handlers, then again it should be straight forward to track down where the problem is occuring. However, using the debugger may not be the best first step if you don't have a good idea where the problem is, such as wrong answers.

I find the best way to isolate these types of problems is via a binary search. Compile half your objects with "-O2" and the other half with "-O0". If the error goes away, then you know the problem is in the group compiled "-O0". If the error persists, then it's in the "-O2" group. Next, compile half of the objects in problem group with "-O0" and the other half with "-O2", and again determine in which group the problem is located. Continue doing this until you are left with a single file.

It does get a bit more complicated if more than file causes the problem, or if there are multiple problems. In this case, I find it easier to compile all objects at "-O0", then re-compile small groups at "-O2" until you encounter a group which fails. You then need to isolate which particular file(s) is causing the problem. Continue until you've found all the problem files.

At this point I would then, if possible, separate out each individual subroutine into it's own file and repeat the binary search until you've gotten it down to a particular subroutine. From here, compile the subroutine with "-O2 -g". You can also compile with "-O0 -Manno" and again with "-O2 -Manno" to compare the assembly files, but more users don't need this level of detail.

This process doesn't work for every problem, but it has help me a lot in tracing down many bugs.

Hope this helps,
Mat
Back to top
View user's profile
Display posts from previous:   
Post new topic   Reply to topic    PGI User Forum Forum Index -> Debugging and Profiling All times are GMT - 7 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © phpBB Group