The Lies of GDB and GCC

Over the past couple of days, I was debugging a race condition, and inserted several sanity checks with calls to abort() within my code to catch where the error was happening. I would start the application in gdb, run until the abort() and then perform a backtrace. Unfortunately, the backtrace was a complete and utter lie, and misled me into wasting a lot of time.

I first discovered this problem when I put a print statement before the place in the code that gdb claimed the backtrace was, and then set a breakpoint on that print statement. The abort() kept happening, but the breakpoint never hit.

I Googled around and eventually found this post, which gave me the key insight to the problem after looking at the assembly dump. It turns out that gcc will treat each abort() as an exit point in the program, and consolidate them into a single block. When gdb hits the abort, it cannot determine which of the original exit points in the source the abort originated from, so it simply guesses one and reports it to be the line of source code that generated the abort(). Here is a simple application that reproduces this behavior.

#include <stdlib.h>
#include <stdio.h>

void foo(int argc) {
    if (argc == 2) {
        abort();
    }
    if (argc == 4) {
        printf("Hello World\n");
        abort();
    }
}
int main(int argc, const char** argv){
    foo(argc);
}

I compiled this using g++ (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609 with the following sequence of commands.

g++ -c -std=c++11 -fPIC -O3 -g -lpthread -o Lie.o Lie.cc
g++ -o Lie Lie.o
objdump -d Lie > Lie.asm

Here is the assembly dump for the function foo().

00000000004005b0 <_Z3fooi>:                                                                    
  4005b0:   48 83 ec 08             sub    $0x8,%rsp                                           
  4005b4:   83 ff 02                cmp    $0x2,%edi                                           
  4005b7:   74 1d                   je     4005d6 <_Z3fooi+0x26>                               
  4005b9:   83 ff 04                cmp    $0x4,%edi                                           
  4005bc:   74 05                   je     4005c3 <_Z3fooi+0x13>                               
  4005be:   48 83 c4 08             add    $0x8,%rsp                                           
  4005c2:   c3                      retq···                                                    
  4005c3:   48 8d 35 9a 00 00 00    lea    0x9a(%rip),%rsi        # 400664 <_IO_stdin_used+0x4>
  4005ca:   bf 01 00 00 00          mov    $0x1,%edi                                           
  4005cf:   31 c0                   xor    %eax,%eax                                           
  4005d1:   e8 aa fe ff ff          callq  400480 <__printf_chk@plt>                           
  4005d6:   e8 85 fe ff ff          callq  400460 <abort@plt>                                  
  4005db:   0f 1f 44 00 00          nopl   0x0(%rax,%rax,1)

If we run this program with a single argument in gdb, gdb will show the following backtrace.

(gdb) bt
#0  0x00007ffff7a43428 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54
#1  0x00007ffff7a4502a in __GI_abort () at abort.c:89
#2  0x00000000004005db in foo (argc=<optimized out>) at Lie.cc:11
#3  0x00000000004004a9 in main (argc=<optimized out>, argv=<optimized out>) at Lie.cc:15

The line Lie.cc:11 is actually under the check if (argc == 4), NOT the earlier check. gdb has lied to us in its ignorance!

This post was last modified on April 04 2017 13:27:13. PST

hq6

A random engineer at Square...