为了账号安全,请及时绑定邮箱和手机立即绑定

GDB分析PHP连接Memcached 导致coredump问题

标签:
PHP

最近一项目压力比较大,502超多,PHP进程coredump比较频繁于是学习了下coredump的调试讲下PHP  coredump的调试,coredump先准备代码比较好,因为PHP是开源的,这点好办。我的是存在/usr/local/src//usr/local/src/php-5.3.10/ 下第一种当然是通用的调试


gdb /usr/local/php/sbin/php-fpm  -c /data/core_files/core-php-fpm-28436-1370008094

第二种是PHP特有的调试获得一个PHP正在运行的进程


#ps -ef|grep php-fpm|head#gdb --pid=$PID>source 

/usr/local/src//usr/local/src/php-5.3.10/.gdbinit

这种方式能调试一些PHP特有的东西,比如包括参数的函数调用栈的信息.先讲通用方式调试


(gdb) bt#0  0x00007ffed0603a75 in raise () from /lib/libc.so.6#1  0x00007ffed06075c0 in abort () from /lib/libc.so.6#2  0x00007ffed342ef7c in memcached_io_slurp (ptr=0x2ef9f50) at libmemcached/io.cc:510#3  0x00007ffed3431da6 in memcached_quit_server (ptr=0x2ef9f50, io_death=false) at libmemcached/quit.cc:95#4  0x00007ffed3431eed in send_quit (ptr=0x2d34e90) at libmemcached/quit.cc:137#5  0x00007ffed342ff39 in _free (ptr=0x2d34e90, release_st=true) at libmemcached/memcached.cc:130#6  0x00007ffed3430397 in memcached_free (ptr=0x2d34e90) at libmemcached/memcached.cc:284#7  0x00000000005d00e0 in php_memc_destroy (m_obj=0x2e3bf60, persistent=0 '\000') at /usr/local/src/php-5.3.10/ext/memcached/php_memcached.c:2519#8  0x00000000005d145f in php_memc_free_storage (i_obj=0x2dea580) at /usr/local/src/php-5.3.10/ext/memcached/php_memcached.c:2530#9  0x000000000081817a in zend_objects_store_del_ref_by_handle_ex (handle=<value optimized out>, handlers=<value optimized out>) at /usr/local/src/php-5.3.10/Zend/zend_objects_API.c:220#10 0x0000000000818193 in zend_objects_store_del_ref (zobject=0x2dea620) at /usr/local/src/php-5.3.10/Zend/zend_objects_API.c:172#11 0x00000000007e9fb5 in _zval_dtor (zval_ptr=<value optimized out>) at /usr/local/src/php-5.3.10/Zend/zend_variables.h:35#12 _zval_ptr_dtor (zval_ptr=<value optimized out>) at /usr/local/src/php-5.3.10/Zend/zend_execute_API.c:447#13 0x00000000008020eb in zend_hash_destroy (ht=0x2d21e20) at /usr/local/src/php-5.3.10/Zend/zend_hash.c:529#14 0x00000000007f5416 in _zval_dtor_func (zvalue=0x2d21df0) at /usr/local/src/php-5.3.10/Zend/zend_variables.c:43#15 0x00000000007e9fb5 in _zval_dtor (zval_ptr=<value optimized out>) at /usr/local/src/php-5.3.10/Zend/zend_variables.h:35#16 _zval_ptr_dtor (zval_ptr=<value optimized out>) at /usr/local/src/php-5.3.10/Zend/zend_execute_API.c:447#17 0x00000000008023f3 in zend_hash_clean (ht=0x2d21cb0) at /usr/local/src/php-5.3.10/Zend/zend_hash.c:561#18 0x00000000007ee2cb in zend_cleanup_op_array_data (function=<value optimized out>) at /usr/local/src/php-5.3.10/Zend/zend_opcode.c:133#19 zend_cleanup_function_data (function=<value optimized out>) at /usr/local/src/php-5.3.10/Zend/zend_opcode.c:140#20 0x0000000000801ea4 in zend_hash_reverse_apply (ht=0x29e5c30, apply_func=0x7ee2a0 <zend_cleanup_function_data>) at /usr/local/src/php-5.3.10/Zend/zend_hash.c:757#21 0x00000000007ea2f3 in shutdown_executor () at /usr/local/src/php-5.3.10/Zend/zend_execute_API.c:296#22 0x00000000007f6563 in zend_deactivate () at /usr/local/src/php-5.3.10/Zend/zend.c:891#23 0x00000000007a47a0 in php_request_shutdown (dummy=<value optimized out>) at /usr/local/src/php-5.3.10/main/main.c:1661#24 0x0000000000884442 in main (argc=<value optimized out>, argv=<value optimized out>) at /usr/local/src/php-5.3.10/sapi/fpm/fpm/fpm_main.c:1886(gdb) f 2#2  0x00007ffed342ef7c in memcached_io_slurp (ptr=0x2ef9f50) at libmemcached/io.cc:510510     libmemcached/io.cc: No such file or directory.        in libmemcached/io.cc

为了调试memcached,会碰到了提示说 libmemcached/io.cc不存在,于是也把libmemcached的源码也下载回来。注意版本得配对,否则容易造成找代码行数找不准,就白瞎了。


warning: Source file is more recent than executable.

看版本用 -i来看


  #/usr/local/php/sbin/php-fpm -i|grep libmemcached  libmemcached version => 1.0.4#wget https://launchpad.net/libmemcached/1.0/1.0.4/+download/libmemcached-1.0.4.tar.gz

把代码解压到php源码目录下  ,如果出错可能是路径不对继续回来gdb


(gdb) f 2#2  0x00007ffed342ef7c in memcached_io_slurp (ptr=0x2ef9f50) at libmemcached/io.cc:510510     in libmemcached/io.cc

看到是libmemcached/io.cc  的510行报错#vim  libmemcached/io.cc +510看到代码如下:


memcached_return_t memcached_io_slurp(memcached_server_write_instance_st ptr

){  assert_msg(ptr, "Programmer error, invalid memcached_server_write_instance_st");  

assert(memcached_is_udp(ptr->root) == false);  

if (ptr->fd == INVALID_SOCKET)  

{    

assert_msg(int(ptr->state) <= int(MEMCACHED_SERVER_STATE_ADDRINFO), "Invalid socket state");   

#line 510    return MEMCACHED_CONNECTION_FAILURE;  }  ssize_t data_read;

这是一个无效的连接? 除非说是网络非常繁忙,或压力很大。往下看看吧


(gdb) f 3#3  0x00007ffed3431da6 in memcached_quit_server (ptr=0x2ef9f50, io_death=false) at libmemcached/quit.cc:95warning: Source file is more recent than executable.95              

while (memcached_continue(rc_slurp= memcached_io_slurp(ptr))) {} ;

是在quit函数中,进代码看看  


WATCHPOINT_ASSERT(rc == MEMCACHED_SUCCESS or rc == MEMCACHED_FETCH_NOTFINISHED);     

/* read until socket is closed, or there is an error      

* closing the socket before all data is read      

* results in server throwing away all data which is      

* not read      

*      

* In .40 we began to only do this if we had been doing buffered      

* requests of had replication enabled.      

*/     

if (ptr->root->flags.buffer_requests or ptr->root->number_of_replicas)     

{      

 memcached_return_t rc_slurp;      

 while (memcached_continue(rc_slurp= memcached_io_slurp(ptr))) {} ;   

#line  95       

WATCHPOINT_ASSERT(rc_slurp == MEMCACHED_CONNECTION_FAILURE);     

}    

 /*      

* memcached_io_read may call memcached_quit_server with io_death if      

* it encounters problems, but we don't care about those occurences.      

* The intention of that loop is to drain the data sent from the      

* server to ensure that the server processed all of the data we      

* sent to the server.      

*/     

ptr->server_failure_counter= 0;   

}   

memcached_io_close(ptr); }

 slurp的意思是:大量读取看看注释的意思:等待socket 正常关闭或没有读完数据但是出现了错误引起的异常关闭,结果是服务器扔掉了所有没有读完的数据。95行的代码是一个条件循环,条件是memcached_io_slurp 的结果继续分析函数  memcached_io_slurp发现是等待读完buffer的过程总体的意思是说服务端开启了buffer,PHP发送了退出请求,所以memcached扩展也被要求退出,然而buffer可能没有读完,所以memcached等待buffer读完,而php等待memcached退出后才能退出


(gdb) f 7#7  0x00000000005d00e0 in php_memc_destroy (m_obj=0x2e3bf60, persistent=0 '\000') at 

/usr/local/src/php-5.3.10/ext/memcached/php_memcached.c:25192519    in /usr/local/src/php-5.3.10/ext/memcached/php_memcached.c一路往后f(gdb) f 24#24 0x0000000000884442 in main (argc=<value optimized out>, argv=<value optimized out>) at /usr/local/src/php-5.3.10/sapi/fpm/fpm/fpm_main.c:18861886                            

php_request_shutdown((void *) 0);

看到是 fpm请求退出进程 php_request_shutdown


fastcgi_request_done:                        

if (primary_script) {                                

efree(primary_script);                        

}                        

if (request_body_fd != -1) {                                

close(request_body_fd);                        

}                        

request_body_fd = -2;                        

if (EG(exit_status) == 255) {                                

if (CGIG(error_header) && *CGIG(error_header)) {                                        

sapi_header_line ctr = {0};                                        

ctr.line = CGIG(error_header);                                        

ctr.line_len = strlen(CGIG(error_header));                                        

sapi_header_op(SAPI_HEADER_REPLACE, &ctr TSRMLS_CC);                                

}                        

}                        

fpm_request_end(TSRMLS_C);                        

fpm_log_write(NULL TSRMLS_CC);                        

STR_FREE(SG(request_info).path_translated);                        

SG(request_info).path_translated = NULL;                        

php_request_shutdown((void *) 0);   #line  1886                        

if (exit_status == 0) {                                

exit_status = EG(exit_status);                        

}                        

requests++;                       

 if (max_requests && (requests == max_requests)) {                                

fcgi_finish_request(&request, 1);                                

if (max_requests != 1) {                                        

/* no need to return exit_status of the last request */                                       

 exit_status = 0;

好像还没有结果。虽然还没查到具体的原因,大概问题归结在memcached扩展和memcached服务器通讯的时候,数据还未传完,PHP进程就要结束,导致coredump。


点击查看更多内容
TA 点赞

若觉得本文不错,就分享一下吧!

评论

作者其他优质文章

正在加载中
  • 推荐
  • 评论
  • 收藏
  • 共同学习,写下你的评论
感谢您的支持,我会继续努力的~
扫码打赏,你说多少就多少
赞赏金额会直接到老师账户
支付方式
打开微信扫一扫,即可进行扫码打赏哦
今天注册有机会得

100积分直接送

付费专栏免费学

大额优惠券免费领

立即参与 放弃机会
微信客服

购课补贴
联系客服咨询优惠详情

帮助反馈 APP下载

慕课网APP
您的移动学习伙伴

公众号

扫描二维码
关注慕课网微信公众号

举报

0/150
提交
取消