201 lines
4.5 KiB
Markdown
201 lines
4.5 KiB
Markdown
|
# 【WMS-UTR项目】tsg_os_container_restart
|
|||
|
|
|
|||
|
|
| ID | Creation Date | Assignee | Status |
|
|||
|
|
|----|----------------|----------|--------|
|
|||
|
|
| OMPUB-1217 | 2024-04-07T17:17:30.000+0800 | 陆秋文 | 已关闭 |
|
|||
|
|
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
根据当地时间4月1日到4月5日的告警，发现多次出现tsg_os_container_restart的告警附件为1号到5号每天的告警统计信息**caoshanfeng** commented on *2024-04-12T09:37:24.776+0800*:
|
|||
|
|
|
|||
|
|
在当地时间4月10日又出现了tsg_os_container_restar的告警，告警内容见附件图片
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
**caoshanfeng** commented on *2024-04-12T10:01:31.788+0800*:
|
|||
|
|
|
|||
|
|
在当地时间4月11日又出现了tsg_os_container_restar的告警，告警内容见附件图片 !4月11日告警.png|thumbnail!
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
**yangwei** commented on *2024-04-15T09:37:25.406+0800*:
|
|||
|
|
|
|||
|
|
* 4月1日-5日重启，现象同https://jira.geedge.net/browse/OMPUB-1196，原因分析为日志量大时包处理线程和后台日志发送线程同时频繁申请和释放内存，可能同时产生大量系统调用，触发watchdog timeout
|
|||
|
|
* 4月9日 全网执行Hotfix[https://docs.geedge.net/pages/viewpage.action?pageId=129087928]，包含修复单向流情况发送空的file chunk，和jemalloc相关参数调整，能够进一步降低日志量和内存申请的系统调用。
|
|||
|
|
* 自4月10日03:25起，出现OMPUB-1225 【WMS-UTR】MSH01和PCAP06存在周期性内存占用上涨导致重启，未能有效评估4月9日更新的效果
|
|||
|
|
* 4月12日观察OMPUB-1225现象消失，至4月13日-14日两天，暂未出现功能端firewall重启的告警，待持续观察，以评估4月9日Hotfix的效果
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
**caoshanfeng** commented on *2024-04-15T10:11:25.040+0800*:
|
|||
|
|
|
|||
|
|
补充4月12日告警与监控信息 !4月12日.png|thumbnail! [^4月12日监控.zip]
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
**caoshanfeng** commented on *2024-04-23T10:48:33.638+0800*:
|
|||
|
|
|
|||
|
|
在当地时间4月22日又出现了tsg_os_container_restar的告警，告警内容与监控信息见附件 !4月22日告警信息.png|thumbnail! [^4月22日监控信息.zip]
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
**caoshanfeng** commented on *2024-04-24T11:30:19.220+0800*:
|
|||
|
|
|
|||
|
|
在当地时间4月23日又出现了tsg_os_container_restar的告警，告警内容与监控信息见附件 !0423告警信息.png|thumbnail! [^0423监控信息.zip] 标红的告警为进行hotfix所导致的无需关注
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
**caoshanfeng** commented on *2024-04-29T10:12:58.256+0800*:
|
|||
|
|
|
|||
|
|
在当地时间4月28日twa-tsgx03出现了tsg_os_container_restar的告警，告警内容与监控信息见附件 !0429告警信息_twa-tsgx03.png|thumbnail! [^twa-tsgx03 (1).html]
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
**caoshanfeng** commented on *2024-05-06T10:51:10.238+0800*:
|
|||
|
|
|
|||
|
|
在当地时间4月30日twa-tsgx02、5月1日msh-tsgx06 出现了tsg_os_container_restar的告警，告警内容与监控信息见附件 !4月30-5月1日告警信息.png|thumbnail! m4 [^5月1日msh-tsgx06 (2).html]
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
**caoshanfeng** commented on *2024-05-08T10:21:43.167+0800*:
|
|||
|
|
|
|||
|
|
在当地时间5月7日twa-tsgx05出现了tsg_os_container_restar的告警，告警内容与监控信息见附件 [^twa-tsgx05 (3).html] !5月7日监控信息.png|thumbnail! [^sapp_process_latency.log.2024-05-07]
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
**caoshanfeng** commented on *2024-05-09T10:49:51.290+0800*:
|
|||
|
|
|
|||
|
|
!5月8日告警信息-2.png|thumbnail! 在当地时间5月8日再次出现了tsg_os_container_restar的告警，告警内容与监控信息见附件（标红的为人为 !5月8日告警信息 重启造成的告警无需关注 [^5月8日监控信息.zip]）
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
**liuyang** commented on *2024-05-10T16:06:14.580+0800*:
|
|||
|
|
|
|||
|
|
后续跟进https://jira.geedge.net/browse/OMPUB-1278，关闭本BUG
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
## Attachments
|
|||
|
|
|
|||
|
|
**56103/0423告警信息.png**
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
**56104/0423监控信息.zip**
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
**56438/0429告警信息_twa-tsgx03.png**
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
**55078/10-11日告警设备对应的监控.zip**
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
**55079/1-5日告警.png**
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
**55080/1-5日告警设备对应的监控.zip**
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
**54975/4月10日告警.png**
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
**54984/4月11日告警.png**
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
**55205/4月12日.png**
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
**55206/4月12日监控.zip**
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
**55987/4月22日告警信息.png**
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
**55988/4月22日监控信息.zip**
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
**56630/4月30-5月1日告警信息.png**
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
**56632/4月30日twa-tsgx02+(2).html**
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
**56634/4月30日twa-tsgx02+(2)-1.html**
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
**56631/5月1日msh-tsgx06+(2).html**
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
**56869/5月7日监控信息.png**
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
**57014/5月8日告警信息.png**
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
**57016/5月8日告警信息-1.png**
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
**57017/5月8日告警信息-2.png**
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
**57015/5月8日监控信息.zip**
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
**56870/sapp_process_latency.log.2024-05-07**
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
**56439/twa-tsgx03+(1).html**
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
**56868/twa-tsgx05+(3).html**
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|