例子

2021-09-29 16:37:17.931204 7fff637f7700 422708 59 ERROR RDMAStack in polling_start

2021-09-29 16:37:17.931227 7fff637f7700 422708 59 ERROR RDMAStack polling_start start  polling thread:1

[New Thread 0x7fff4475f700 (LWP 423826)]

2021-09-29 16:37:17.937471 7fff637f7700 422708 59 ERROR RDMAStack polling_start start: 1 handle_rx_fun  threads

2021-09-29 16:37:17.937488 7fff637f7700 422708 59 ERROR RDMAStack polling_start start: handle_rx_fun  thread 

[New Thread 0x7fff43f5e700 (LWP 423827)]

2021-09-29 16:37:17.938410 7fff4475f700 423826 18 ERROR RDMAStack max_num:1 bf_num:

2021-09-29 16:37:17.938437 7fff4475f700 423826 18 ERROR RDMAStack busy_polling bf_num: tail:

terminate called without an active exception

2021-09-29 16:37:17.938455 7fff4475f700 423826 18 ERROR RDMAStack max_num:1 bf_num: tail:

2021-09-29 16:37:17.938459 7fff4475f700 423826 18 ERROR RDMAStack busy_polling bf_num: tail:

2021-09-29 16:37:17.938464 7fff4475f700 423826 18 ERROR RDMAStack max_num:1 bf_num: tail:

2021-09-29 16:37:17.938467 7fff4475f700 423826 18 ERROR RDMAStack busy_polling bf_num: tail:

2021-09-29 16:37:17.938473 7fff4475f700 423826 18 ERROR RDMAStack max_num:1 bf_num: tail:

2021-09-29 16:37:17.938476 7fff4475f700 423826 18 ERROR RDMAStack busy_polling bf_num: tail:

2021-09-29 16:37:17.938481 7fff4475f700 423826 18 ERROR RDMAStack max_num:1 bf_num: tail:

2021-09-29 16:37:17.938484 7fff4475f700 423826 18 ERROR RDMAStack busy_polling bf_num: tail:

2021-09-29 16:37:17.938494 7fff4475f700 423826 18 ERROR RDMAStack max_num:1 bf_num: tail:

2021-09-29 16:37:17.938498 7fff4475f700 423826 18 ERROR RDMAStack busy_polling bf_num: tail:

Program received signal SIGABRT, Aborted.

[Switching to Thread 0x7fff637f7700 (LWP 422708)]

0x00007fffeb4b3387 in raise () from /lib64/libc.so.6

上面表示 程序abort了。abort之前在线程7fff4475f700运行,abort之后程序切换到0x7fff637f7700线程。

(gdb) i  threads 查看线程信息

查看到线程7fff4475f700的 id 是79,而当前在线程12 (前面有*号)


(gdb) i threads
  Id   Target Id         Frame 
  80   Thread 0x7fff43f5e700 (LWP 423827) "io-workerb-0" 0x00007fffefaafc98 in LC_GetThreadIndex (threadId=423827)
    at /home/lcx/ceph-L/src/umm/localcache.c:41
  79   Thread 0x7fff4475f700 (LWP 423826) "io-workerb-0" 0x00007fffeb570ccd in poll () from /lib64/libc.so.6
  78   Thread 0x7fff4a05c700 (LWP 423816) "ms_local" 0x00007fffeca95a35 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/.0
  77   Thread 0x7fff4a85d700 (LWP 423815) "ms_dispatch" 0x00007fffeca95a35 in 

……

  13   Thread 0x7fff62ff6700 (LWP 422709) "io-workerb-1" 0x00007fffeb57bfd3 in epoll_wait () from /lib64/libc.so.6
* 12   Thread 0x7fff637f7700 (LWP 422708) "io-workerb-0" 0x00007fffeb4b3387 in raise () from /lib64/libc.so.6
  11   Thread 0x7fff63ff8700 (LWP 422702) "io-worker-2" 0x00007fffeb57bfd3 in epoll_wait () from 
 


(gdb) thread 79 切换到线程79

(gdb) thread 79

[Switching to thread 79 (Thread 0x7fff4475f700 (LWP 423826))]

#0  0x00007fffeb570ccd in poll () from /lib64/libc.so.6

(gdb) bt 查看线程 79的调用堆栈 

(gdb) bt

#0  0x00007fffeb570ccd in poll () from /lib64/libc.so.6

#1  0x00007fffec815c3b in co::libgo_poll (fds=0x7fff4475d2b0, nfds=2, timeout=200, nonblocking_check=true)

    at /compiledir/zhangtao/workspace/Libgo/libgo/netio/unix/hook.cpp:84

#2  0x00007fffec80ccdc in poll (fds=0x7fff4475d2b0, nfds=2, timeout=200)

    at /compiledir/zhangtao/workspace/Libgo/libgo/netio/unix/hook.cpp:477

#3  0x00005555562b24cb in RDMADispatcher::busy_polling() ()

#4  0x00007fffebe18360 in ?? () from /opt/h3c/lib/libstdc++.so.6

#5  0x00007fffeca91ea5 in start_thread () from /lib64/.0

#6  0x00007fffeb57b9fd in clone () from /lib64/libc.so.6

(gdb) f 3

#3  0x00005555562b24cb in RDMADispatcher::busy_polling() ()

(gdb) i local

No symbol table info available.

(gdb) f 2

#2  0x00007fffec80ccdc in poll (fds=0x7fff4475d2b0, nfds=2, timeout=200)

    at /compiledir/zhangtao/workspace/Libgo/libgo/netio/unix/hook.cpp:477

477     /compiledir/zhangtao/workspace/Libgo/libgo/netio/unix/hook.cpp: 没有那个文件或目录.

(gdb) i local

tk = 0x0

__FUNCTION__ = "poll"

(gdb) i arg

fds = 0x7fff4475d2b0

nfds = 2

timeout = 200

(gdb) 

线程的查看以及利用gdb调试多线程

命令行查看:

//查看当前运行的进程
ps aux|grep a.out
//查看当前运行的轻量级进程
ps -aL|grep a.out
//查看主线程和新线程的关系
pstree -p 主线程id

【GDB】GDB多线程查看崩溃时线程的堆栈线程的查看以及利用gdb调试多线程_#include 

2. 线程栈结构的查看

1. 获取线程ID
2. 通过命令查看栈结构 ps stack 线程ID

【GDB】GDB多线程查看崩溃时线程的堆栈线程的查看以及利用gdb调试多线程_多线程_02

3. 利用gdb查看线程信息

  1. 将进程附加到gdb调试器当中,查看是否创建了新线程:gdb attach 主线程ID 

【GDB】GDB多线程查看崩溃时线程的堆栈线程的查看以及利用gdb调试多线程_自动驾驶_03 

2.查看线程的一些信息

//1.查看进程:info inferiors
//2.查看线程:info threads
//3.查看线程栈结构:bt
//4.切换线程:thread n(n代表第几个线程)

【GDB】GDB多线程查看崩溃时线程的堆栈线程的查看以及利用gdb调试多线程_html5_04

 

4. 利用gdb调试多线程

  当程序没有启动,线程还没有执行,此时利用gdb调试多线程和调试普通程序一样,通过设置断点,运行,查看信息等等,在这里不在演示,最后会加上调试线程的命令

  1. 设置断点
    //1. 设置断点:break 行号/函数名//2. 查看断点:info b

 【GDB】GDB多线程查看崩溃时线程的堆栈线程的查看以及利用gdb调试多线程_自动驾驶_05

 

执行线程2的函数,指行完毕继续运行到断点处

1. 继续使某一线程运行:thread apply 1-n(第几个线程) n
2. 重新启动程序运行到断点处:r

 【GDB】GDB多线程查看崩溃时线程的堆栈线程的查看以及利用gdb调试多线程_html5_06

只运行当前线程

1. 设置:set scheduler-locking on
2. 运行:n

 【GDB】GDB多线程查看崩溃时线程的堆栈线程的查看以及利用gdb调试多线程_自动驾驶_07

所有线程并发执行

    1. 设置:set scheduler-locking off
2. 运行:n

【GDB】GDB多线程查看崩溃时线程的堆栈线程的查看以及利用gdb调试多线程_html5_08

 总结调试多线程的命令

【GDB】GDB多线程查看崩溃时线程的堆栈线程的查看以及利用gdb调试多线程_#include_09

 

命令                        用法

info threads    显示当前可调试的所有线程,每个线程会有一个GDB为其分配的ID,后面操作线程的时候会用到这个ID。 前面有*的是当前调试的线程

thread ID(1,2,3…)    切换当前调试的线程为指定ID的线程

break thread_test.c:123 thread all(例:在相应函数的位置设置断点break pthread_run1)    在所有线程中相应的行上设置断点

thread apply ID1 ID2 command    让一个或者多个线程执行GDB命令command

thread apply all command    让所有被调试线程执行GDB命令command

set scheduler-locking 选项 command    设置线程是以什么方式来执行命令

set scheduler-locking off    不锁定任何线程,也就是所有线程都执行,这是默认值

set scheduler-locking on    只有当前被调试程序会执行

set scheduler-locking on step    在单步的时候,除了next过一个函数的情况(熟悉情况的人可能知道,这其实是一个设置断点然后continue的行为)以外,只有当前线程会执行

 

 示例代码

#include <stdio.h>
#include <unistd.h>
#include <pthread.h>
#include <stdlib.h>
#include <string.h>

void* pthread_run1(void* arg)
{
(void)arg;

while(1)
{
printf("I am thread1,ID: %d\n",pthread_self());
sleep(1);
}
}

void* pthread_run2(void* arg)
{
(void)arg;

while(1)
{
printf("I am thread2,ID: %d\n",pthread_self());
sleep(1);
}
}


int main()
{

pthread_t tid1;
pthread_t tid2;

pthread_create(&tid1,NULL,pthread_run1,NULL);
pthread_create(&tid2,NULL,pthread_run2,NULL);

printf("I am main thread\n");

pthread_join(tid1,NULL);
pthread_join(tid2,NULL);
return 0;
}