91 lines
2.6 KiB
Markdown
91 lines
2.6 KiB
Markdown
|
# 【E21现场】多块NPB出现tsg_9140_service_sapp_running告警
|
|||
|
|
|
|||
|
|
| ID | Creation Date | Assignee | Status |
|
|||
|
|
|----|----------------|----------|--------|
|
|||
|
|
| OMPUB-893 | 2023-04-10T21:49:46.000+0800 | 杨威 | 完成 |
|
|||
|
|
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
查询过去两天的告警，tsg_9140_service_sapp_running告警共出现10次：
|
|||
|
|
BJR-PE-NPB01 10.243.11.1 出现1次；
|
|||
|
|
|
|||
|
|
OAP-PE-NPB01 10.231.11.1 出现5次；
|
|||
|
|
|
|||
|
|
OAP-PE-NPB04 10.231.11.4 出现1次；
|
|||
|
|
|
|||
|
|
SSM-IGW-NPB02 10.226.11.2 出现1次；
|
|||
|
|
|
|||
|
|
SSM-IGW-NPB05 10.226.11.5 出现1次；
|
|||
|
|
|
|||
|
|
SSM-IGW-NPB06 10.226.11.6 出现1次。
|
|||
|
|
|
|||
|
|
附件为相关core文件、Alerts以及出现次数较多的NPB 10.231.11.1过去七天Assets。**yangwei** commented on *2023-04-15T16:25:13.264+0800*:
|
|||
|
|
|
|||
|
|
现场传回来的core，未见如何产生的描述，看大小推测为minidump转换成的coredump文件，目前不包含有效符号名
|
|||
|
|
|
|||
|
|
建议附上出现异常重启的NPB如下信息：
|
|||
|
|
|
|||
|
|
coredumpctl info
|
|||
|
|
//在出现段错误时的coredump的栈信息
|
|||
|
|
|
|||
|
|
journalctl -r -u sapp
|
|||
|
|
//systemd中记录的重启时原因
|
|||
|
|
|
|||
|
|
OAP-PE-NPB01 10.231.11.1 ，按现场群中提供的操作，关闭dtls功能后继续观察
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
**chengsiyuan** commented on *2023-04-17T16:36:38.686+0800*:
|
|||
|
|
|
|||
|
|
附件minidump+coredump+systemd.zip包含出现异常重启NPB当天的minidump文件以及coredumpctl info截图和journalctl -r -u sapp信息；
|
|||
|
|
|
|||
|
|
OAP-PE 10.231.11.1于2023-04-11 09:57更新后，在2023-04-15 15:52:21又出现tsg_9140_service_sapp_running告警，并产生了core文件，附件为OAP-PE-10.231.11.1_Arp15.zip包含重启NPB当天的minidump文件、core文件以及出现告警当天的coredumpctl info截图和journalctl -r -u sapp信息
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
**chengsiyuan** commented on *2023-05-25T14:50:23.382+0800*:
|
|||
|
|
|
|||
|
|
从2023-5-15到目前为止，OAP-PE-NPB01 10.231.11.1未出现重启现象
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
**chengsiyuan** commented on *2023-05-31T22:09:26.566+0800*:
|
|||
|
|
|
|||
|
|
截至到目前为止，OAP-PE-NPB01 10.231.11.1未出现重启现象。
|
|||
|
|
|
|||
|
|
目前还有其他NPB出现tsg_9140_service_sapp_running告警，是否需要再挑一台NPB执行上次提供的操作：
|
|||
|
|
|
|||
|
|
关闭DTLS功能，操作如下：
|
|||
|
|
1、修改/opt/tsg/sapp/plug/conflist.inf文件，用#注释掉./plug/protocol/dtls/dtls.inf和./plug/business/fw_dtls_plug/fw_dtls_plug.inf这两行
|
|||
|
|
2、修改/opt/tsg/sapp/tsgconf/main.conf文件，将[SYSTEM]下IDENTIFY_PROTO_NAME="DNS;QUIC;HTTP;MAIL;FTP;SSL;RTP;SSH;RADIUS;SOCKS;STRATUM;RDP;DTLS;GTPC;"中的DTLS;删除
|
|||
|
|
|
|||
|
|
重启sapp服务
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
## Attachments
|
|||
|
|
|
|||
|
|
**37049/20230410_core.zip**
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
**37411/minidump+coredump+systemd.zip**
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
**37412/OAP-PE-10.231.11.1_Arp15.zip**
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|