福建项目：LUA虚拟机线程不安全，多线程执行LUA脚本导致程序重启(两个线程分别加载LUA和执行LUA)

ID	Creation Date	Assignee	Status
OMPUB-937	2023-05-31T11:33:50.000+0800	刘学利	已解决

操作步骤： 1、新建modbus属性，添加相关lua脚本； 2、新建modbus特征，添加【tcp-port：0-65535】and【tcp-port not 443】and【modbus属性】； 3、编辑自定义app，删除旧modbus特征，引用新modbus特征；

[~wangjunhao] 补充一下现场细节： 1、新建自定义属性截图； 2、新建自定义特征截图； 3、自定义app截图； 4、lua文件； 5、其他相关文件liuxueli commented on 2023-05-31T18:35:31.673+0800:

192.168.18.11机器，使用dmesg -H命令查看重启记录： ** 2023-03-31,2023-04-24由于http.so库段错误导致重启 ** 2023-05-31由于libelua库段错误导致重启 ** {code:java} [ +0.014589] mlx5_core 0000:17:00.1 ens5f1: S-tagged traffic will be dropped while C-tag vlan stripping is enabled [ +0.181292] device ens5f1 left promiscuous mode [Mar31 18:11] sapp_marsio_46[9547]: segfault at 7fef66c9a000 ip 00007ff8f61047cb sp 00007ff5491cd320 error 4 in http.so[7ff8f60fd000+18000] [Apr12 14:44] device lo entered promiscuous mode [ +0.524534] device lo left promiscuous mode [Apr12 14:45] device lo entered promiscuous mode [ +3.862364] device lo left promiscuous mode [ +4.152328] device lo entered promiscuous mode [ +0.021275] device lo left promiscuous mode [Apr12 14:46] device lo entered promiscuous mode [ +0.024346] device lo left promiscuous mode [Apr12 14:47] device lo entered promiscuous mode [ +0.074526] device lo left promiscuous mode [Apr14 01:37] device lo entered promiscuous mode [ +1.290589] device lo left promiscuous mode [ +7.802062] device lo entered promiscuous mode [ +1.169179] device lo left promiscuous mode [Apr14 11:10] device lo entered promiscuous mode [ +0.028194] device lo left promiscuous mode [Apr14 15:10] device lo entered promiscuous mode [ +0.023993] device lo left promiscuous mode [Apr24 07:09] sapp_marsio_82[51688]: segfault at 7ff11da40000 ip 00007ff8943047cb sp 00007ff6423a2320 error 4 in http.so[7ff8942fd000+18000] [Apr26 14:03] device lo entered promiscuous mode [Apr26 14:15] device lo left promiscuous mode [Apr26 14:39] IPv6: ADDRCONF(NETDEV_UP): eno2: link is not ready [Apr26 14:40] mlx5_core 0000:17:00.1 ens5f1: Link up [ +0.000944] IPv6: ADDRCONF(NETDEV_UP): ens5f1: link is not ready [ +0.001005] IPv6: ADDRCONF(NETDEV_CHANGE): ens5f1: link becomes ready [May18 00:27] mlx5_core 0000:b1:00.0 ens7f0: Link down [May18 00:41] mlx5_core 0000:b1:00.0 ens7f0: Link up [May18 00:46] mlx5_core 0000:b1:00.0 ens7f0: Link down [May18 00:53] mlx5_core 0000:b1:00.0 ens7f0: Link up [May26 16:24] hrtimer: interrupt took 30566 ns [May30 23:25] device lo entered promiscuous mode [ +2.307671] device lo left promiscuous mode [ +1.814181] device lo entered promiscuous mode [May30 23:33] device lo left promiscuous mode [May31 10:31] traps: MAAT_APP_SIGNAT[51368] general protection ip:7ff65b84c199 sp:7ff65a3fb9c0 error:0 in libelua.so[7ff65b82a000+80000] {code}

liuxueli commented on 2023-05-31T18:52:57.454+0800:

192.168.21.3机器，minidump栈信息：

** {code:java} warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available. Failed to read a valid object file image from memory. Core was generated by `/opt/tsg/sapp/sapp -c /opt/tsg/sapp/etc/sapp.toml -C /opt/tsg/sapp -D /opt/tsg/'. #0 0x00007ff889090e24 in lj_alloc_malloc () from /opt/tsg/framework/lib/libelua.so Missing separate debuginfos, use: debuginfo-install sapp-4.2.90.8c77537-1.el7.x86_64 (gdb) bt #0 0x00007ff889090e24 in lj_alloc_malloc () from /opt/tsg/framework/lib/libelua.so #1 0x00007ff88909ac0e in lj_mem_realloc () from /opt/tsg/framework/lib/libelua.so #2 0x00007ff88907d4af in lj_func_newL_gc () from /opt/tsg/framework/lib/libelua.so #3 0x00007ff8890961f9 in lj_BC_FNEW () from /opt/tsg/framework/lib/libelua.so #4 0x00007ff889085972 in lua_pcall () from /opt/tsg/framework/lib/libelua.so #5 0x00007ff88907af15 in elua_call_script (errmsg=0x7ffd83932c14 <Address 0x7ffd83932c14 out of bounds>, script_id=, L=0x7fff03e1e378) at /usr/src/debug/elua-2.0.1.7760c27-Linux/LIBRARIES/src_0/src/elua_func.cpp:1071 #6 elua_execute_script (escript=0x7ffff35ce4e0, input=, input_len=388, userdata=, ctx=, output=0x7ff7023f8eb0) at /usr/src/debug/elua-2.0.1.7760c27-Linux/LIBRARIES/src_0/src/elua_func.cpp:1192 #7 0x00007ff889302ade in app_call_lua_script(streaminfo*, elua_vm*, lua_rt_info*, char*, unsigned long, extract_attribute*, int) () from ./plug/business/app_sketch_local/app_sketch_local.so #8 0x00007ff8892fc801 in identify_app_by_user_define_attributes(streaminfo*, local_context*, Maat_rule_t*, int, int) [clone .constprop.8] () from ./plug/business/app_sketch_local/app_sketch_local.so #9 0x00007ff8892fe0ea in identify_app_by_tcp_payload(streaminfo*, local_context*, int) () from ./plug/business/app_sketch_local/app_sketch_local.so #10 0x00007ff8892fe46c in APP_SKETCH_TCP_PLUG_ENTRY () from ./plug/business/app_sketch_local/app_sketch_local.so #11 0x000000000044b717 in plugin_call_streamentry (type=2, pFunInfo=0x7ff33f32a8d8, a_stream=0x7fffac74c468, transport_hdr=0x0, thread_seq=0, a_packet=0x7ff04c6b3b0a) at /tmp/46e581ef-8a14-42d5-ba44-bcb8d209cbbe/sapp-v4.2.90/src/dealpkt/callapp.c:305 #12 0x000000000044bd70 in call_streamentry (a_stream=0x7fffac74c468, this_iphdr=0x7ff04c6b3b0a, transport_hdr=0x7ff04c6b3b1e, raw_pkt=0x7fef6c359f78, pFunInfo=0x7ff33f32a8d8) at /tmp/46e581ef-8a14-42d5-ba44-bcb8d209cbbe/sapp-v4.2.90/src/dealpkt/callapp.c:457 #13 0x000000000044be4b in stream_state_pending (a_stream=0x7fffac74c468, this_iphdr=0x7ff04c6b3b0a, transport_hdr=0x7ff04c6b3b1e, raw_pkt=0x7fef6c359f78, funnum=10, pfunAarry=0x90afa0 <g_StreamTcpFun>, apme=0x7ff5b6919820, opstate=0x7fffac74c484 <Address 0x7fffac74c484 out of bounds>, smart_offload_flag=1 '\001') at /tmp/46e581ef-8a14-42d5-ba44-bcb8d209cbbe/sapp-v4.2.90/src/dealpkt/callapp.c:481 #14 0x000000000044c36b in stream_process (a_stream=0x7fffac74c468, this_iphdr=0x7ff04c6b3b0a, transport_hdr=0x7ff04c6b3b1e, raw_pkt=0x7fef6c359f78, funnum=10, pfunAarry=0x90afa0 <g_StreamTcpFun>, apme=0x7ff5b6919820, opstate=0x7fffac74c484 <Address 0x7fffac74c484 out of bounds>, smart_offload_flag=1 '\001') at /tmp/46e581ef-8a14-42d5-ba44-bcb8d209cbbe/sapp-v4.2.90/src/dealpkt/callapp.c:721 #15 0x000000000044c67f in stream_process_tcp (a_tcp=0x7fffac74c468, this_iphdr=0x7ff04c6b3b0a, transport_hdr=0x7ff04c6b3b1e, raw_pkt=0x7fef6c359f78, apme=0x7ff5b6919820, popstate=0x7fffac74c484 <Address 0x7fffac74c484 out of bounds>) at /tmp/46e581ef-8a14-42d5-ba44-bcb8d209cbbe/sapp-v4.2.90/src/dealpkt/callapp.c:817 #16 0x000000000045399d in tcp_deal_unorder (pindex=0x7fffac74c410, a_tcp=0x7fffac74c468, rcv=0x7ff54a555498, snd=0x7ff54abf46f8, orderflag=0) at /tmp/46e581ef-8a14-42d5-ba44-bcb8d209cbbe/sapp-v4.2.90/src/dealpkt/deal_tcp.c:2180 #17 0x0000000000451e21 in tcp_free_stream (pindex=0x7fffac74c410, this_ip_hdr=0x0, transport_hdr=0x0, raw_pkt=0x0) at /tmp/46e581ef-8a14-42d5-ba44-bcb8d209cbbe/sapp-v4.2.90/src/dealpkt/deal_tcp.c:1354 #18 0x00000000004622cc in del_stream_by_time (plist=0x7ffff23f9788, current_drive_index=0x7fffabd90010) at /tmp/46e581ef-8a14-42d5-ba44-bcb8d209cbbe/sapp-v4.2.90/src/dealpkt/stream_manage.c:465 #19 0x00000000004623c9 in lrustream (pindex=0x7fffabd90010) at /tmp/46e581ef-8a14-42d5-ba44-bcb8d209cbbe/sapp-v4.2.90/src/dealpkt/stream_manage.c:506 #20 0x00000000004552f1 in deal_tcp_stream (pindex=0x7ff7023fa060, this_iphdr=0x7f434be4f722, this_tcphdr=0x7f434be4f74a, tcplen=1288, raw_pkt=0x7ff7023fb310, offset_to_raw_pkt_hdr=122) at /tmp/46e581ef-8a14-42d5-ba44-bcb8d209cbbe/sapp-v4.2.90/src/dealpkt/deal_tcp.c:2946 #21 0x0000000000455a3b in dealtcppkt (pfindex=0x7ff7023fa060, this_iphdr=0x7f434be4f722, this_tcphdr=0x7f434be4f74a, thread_num=0, routedir=0 '\000', tcpdatalen=1288, raw_packet=0x7ff7023fb310, offset_to_raw_pkt_hdr=122) at /tmp/46e581ef-8a14-42d5-ba44-bcb8d209cbbe/sapp-v4.2.90/src/dealpkt/deal_tcp.c:3118 #22 0x0000000000455bd0 in dealipv6tcppkt (pfindex=0x7ff7023fa060, this_iphdr=0x7f434be4f722, data=0x7f434be4f74a <Address 0x7f434be4f74a out of bounds>, datalen=1288, thread_num=0, routedir=0 '\000', raw_pkt=0x7ff7023fb310, offset_to_raw_pkt_hdr=122) at /tmp/46e581ef-8a14-42d5-ba44-bcb8d209cbbe/sapp-v4.2.90/src/dealpkt/deal_tcp.c:3172 #23 0x000000000045c576 in ipv6_entry (pfstream_pr=0x7ff7023fa288, this_layer_hdr=0x7f434be4f722, thread_num=0, routedir=0 '\000', raw_pkt=0x7ff7023fb310, offset_to_raw_pkt_hdr=82) at /tmp/46e581ef-8a14-42d5-ba44-bcb8d209cbbe/sapp-v4.2.90/src/dealpkt/deal_ipv6.c:848 #24 0x000000000046bb96 in gtp_entry (pfstream_pr=0x7fffac8d8a68, this_layer_data=0x7f434be4f70a, thread_num=0, routedir=0 '\000', raw_pkt=0x7ff7023fb310, offset_to_raw_pkt_hdr=66) at /tmp/46e581ef-8a14-42d5-ba44-bcb8d209cbbe/sapp-v4.2.90/src/dealpkt/deal_gprs_tunnel.c:176 #25 0x0000000000460d1f in dealipv6udppkt (pindex=0x7ff7023fa4a0, a_packet=0x7f434be4f6e2, data=0x7f434be4f70a <Address 0x7f434be4f70a out of bounds>, thread_num=0, routedir=0 '\000', raw_pkt=0x7ff7023fb310, offset_to_raw_pkt_hdr=58) at /tmp/46e581ef-8a14-42d5-ba44-bcb8d209cbbe/sapp-v4.2.90/src/dealpkt/deal_udp.c:940 #26 0x000000000045c60c in ipv6_entry (pfstream_pr=0x7ff7023fa728, this_layer_hdr=0x7f434be4f6e2, thread_num=0, routedir=0 '\000', raw_pkt=0x7ff7023fb310, offset_to_raw_pkt_hdr=18) at /tmp/46e581ef-8a14-42d5-ba44-bcb8d209cbbe/sapp-v4.2.90/src/dealpkt/deal_ipv6.c:862 #27 0x000000000046a378 in IEEE_8021_entry (pfstream_pr=0x7ff7023fa8a8, this_layer_data=0x7f434be4f6de, thread_num=0, routedir=0 '\000', raw_pkt=0x7ff7023fb310, offset_to_raw_pkt_hdr=14, eth_type=33024) at /tmp/46e581ef-8a14-42d5-ba44-bcb8d209cbbe/sapp-v4.2.90/src/dealpkt/deal_vlan.c:111 #28 0x000000000045e089 in eth_entry (fstream_pr=0x0, this_layer_hdr=0x7f434be4f6d0, thread_num=0, dir=0 '\000', raw_pkt=0x7ff7023fb310, offset_to_raw_pkt_hdr=0) at /tmp/46e581ef-8a14-42d5-ba44-bcb8d209cbbe/sapp-v4.2.90/src/dealpkt/deal_ethernet.c:189 #29 0x0000000000442238 in mesa_default_pkt_cb (p_raw_pkt=0x7ff7023fb310, dir=0 '\000', thread_num=0) at /tmp/46e581ef-8a14-42d5-ba44-bcb8d209cbbe/sapp-v4.2.90/src/packet_io/packet_io.c:649 #30 0x00000000005387cb in marsio4_pkt_hand (tid=0, rx_buff=0x7f434be4f540, raw_pkt=0x7ff7023fb310, dir=0 '\000') at /tmp/46e581ef-8a14-42d5-ba44-bcb8d209cbbe/sapp-v4.2.90/src/packet_io/packet_io_marsio.c:664 #31 0x00000000005388d3 in marsio4_process_packet (tid=0, raw_pkt=0x7ff7023fb310) at /tmp/46e581ef-8a14-42d5-ba44-bcb8d209cbbe/sapp-v4.2.90/src/packet_io/packet_io_marsio.c:706 #32 0x0000000000539074 in marsio4_worker (arg=0x3a88320 <marsio4_work_thread_seq>) at /tmp/46e581ef-8a14-42d5-ba44-bcb8d209cbbe/sapp-v4.2.90/src/packet_io/packet_io_marsio.c:872 #33 0x00007ffff79ade25 in start_thread () from /lib64/libpthread.so.0 #34 0x00007ffff5c0a34d in clone () from /lib64/libc.so.6 {code}

zhangzhihan commented on 2023-06-15T10:42:30.580+0800:

[~yangwei] 现场测试环境测试结果如下： 1、两次上传完全一致的modbus lua脚本 → sapp重启； 2、两次上传内容不一致的modbus lua脚本 → sapp重启； 3、第一次上传modbus lua脚本，第二次上传其他任意lua脚本 → sapp不重启； 4、两次上传完全一致的其他lua脚本 → sapp不重启； 5、第一次上传modbusA脚本，删除A，再上传modbusB脚本 → sapp不重启

补充： 2用到的脚本如下 [^modbus_20230517.lua] [^modbusall(1)(1).lua]

4用到的脚本如下 [^tesst.lua]

gitlab commented on 2023-06-19T15:11:50.910+0800:

[刘学利|https://git.mesalab.cn/liuxueli] mentioned this issue in [a commit|b89bf8fc09] of [AppSketch / app_sketch_local|https://git.mesalab.cn/appsketch/app_sketch_local] on branch [bugfix-segmentfault-lua-cache-OMPUB-937|https://git.mesalab.cn/appsketch/app_sketch_local/-/tree/bugfix-segmentfault-lua-cache-OMPUB-937]:{quote}OMPUB-937: LUA虚拟机线程不安全, LUA脚本执行前进行cache{quote}

gitlab commented on 2023-06-19T15:38:35.946+0800:

[刘学利|https://git.mesalab.cn/liuxueli] mentioned this issue in [a commit|11225c7384] of [AppSketch / app_sketch_local|https://git.mesalab.cn/appsketch/app_sketch_local] on branch [bugfix-segmentfault-lua-cache-OMPUB-937|https://git.mesalab.cn/appsketch/app_sketch_local/-/tree/bugfix-segmentfault-lua-cache-OMPUB-937]:{quote}OMPUB-937: LUA虚拟机线程不安全, 命中前置特征后LUA脚本执行前进行cache{quote}

gitlab commented on 2023-06-19T16:32:50.912+0800:

[刘学利|https://git.mesalab.cn/liuxueli] mentioned this issue in [a commit|d8d2c6af6c] of [AppSketch / app_sketch_local|https://git.mesalab.cn/appsketch/app_sketch_local] on branch [bugfix-segmentfault-lua-cache-OMPUB-937|https://git.mesalab.cn/appsketch/app_sketch_local/-/tree/bugfix-segmentfault-lua-cache-OMPUB-937]:{quote}OMPUB-937: LUA虚拟机线程不安全, 命中前置特征后LUA脚本执行前进行cache{quote}

gitlab commented on 2023-06-20T15:49:59.198+0800:

[刘学利|https://git.mesalab.cn/liuxueli] mentioned this issue in [a commit|cc02934974] of [AppSketch / app_sketch_local|https://git.mesalab.cn/appsketch/app_sketch_local] on branch [bugfix-segmentfault-lua-cache-OMPUB-937-v23.06|https://git.mesalab.cn/appsketch/app_sketch_local/-/tree/bugfix-segmentfault-lua-cache-OMPUB-937-v23.06]:{quote}OMPUB-937: LUA虚拟机线程不安全, 命中前置特征后LUA脚本执行前进行cache{quote}

gitlab commented on 2023-06-30T18:32:04.975+0800:

[刘学利|https://git.mesalab.cn/liuxueli] mentioned this issue in [a merge request|https://git.mesalab.cn/appsketch/app_sketch_local/-/merge_requests/70] of [AppSketch / app_sketch_local|https://git.mesalab.cn/appsketch/app_sketch_local] on branch [bugfix-segmentfault-lua-cache-OMPUB-937-v23.06|https://git.mesalab.cn/appsketch/app_sketch_local/-/tree/bugfix-segmentfault-lua-cache-OMPUB-937-v23.06]:{quote}OMPUB-937: LUA虚拟机线程不安全, 命中前置特征后LUA脚本执行前进行cache{quote}

gitlab commented on 2023-06-30T18:59:03.964+0800:

[刘学利|https://git.mesalab.cn/liuxueli] mentioned this issue in [a commit|d49dbbaded] of [TSG / tsg-os-buildimage|https://git.mesalab.cn/tsg/tsg-os-buildimage] on branch [update-v23.07-app-sketch|https://git.mesalab.cn/tsg/tsg-os-buildimage/-/tree/update-v23.07-app-sketch]:{quote}OMPUB-937: 福建项目：LUA虚拟机线程不安全，多线程执行LUA脚本导致程序重启(两个线程分别加载LUA和执行LUA){quote}

gitlab commented on 2023-06-30T19:07:43.894+0800:

[刘学利|https://git.mesalab.cn/liuxueli] mentioned this issue in [a merge request|https://git.mesalab.cn/tsg/tsg-os-buildimage/-/merge_requests/1493] of [TSG / tsg-os-buildimage|https://git.mesalab.cn/tsg/tsg-os-buildimage] on branch [update-v23.07-app-sketch|https://git.mesalab.cn/tsg/tsg-os-buildimage/-/tree/update-v23.07-app-sketch]:{quote}OMPUB-937: 福建项目：LUA虚拟机线程不安全，多线程执行LUA脚本导致程序重启(两个线程分别加载LUA和执行LUA){quote}

gitlab commented on 2023-06-30T19:17:20.497+0800:

[付明卫|https://git.mesalab.cn/fumingwei] mentioned this issue in [a commit|32776525e7] of [TSG / tsg-os-buildimage|https://git.mesalab.cn/tsg/tsg-os-buildimage] on branch [update-v23.07-app-sketch|https://git.mesalab.cn/tsg/tsg-os-buildimage/-/tree/update-v23.07-app-sketch]:{quote}OMPUB-937: 福建项目：LUA虚拟机线程不安全，多线程执行LUA脚本导致程序重启(两个线程分别加载LUA和执行LUA){quote}