64 lines
1.7 KiB
Markdown
64 lines
1.7 KiB
Markdown
|
# 【E21现场】频繁出现Minio-Sync-Status及CM-Sync-Error告警
|
|||
|
|
|
|||
|
|
| ID | Creation Date | Assignee | Status |
|
|||
|
|
|----|----------------|----------|--------|
|
|||
|
|
| OMPUB-753 | 2022-12-23T14:48:55.000+0800 | 雷军 | 已关闭 |
|
|||
|
|
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
现场近期频繁出现Minio-Sync-Status及CM-Sync-Error告警以及CM_Branch_Redis**zhangwei** commented on *2022-12-23T15:17:59.915+0800*:
|
|||
|
|
|
|||
|
|
# 先通过界面将丢失Minio文件的中间证书缓存Disabled，恢复数据同步。
|
|||
|
|
# [~leijun] 排查Minio文件丢失原因
|
|||
|
|
# 可通过SQL脚本，删除丢失Minio文件的中间证书缓存数据，看后续是否可重新学习到这条中间证书缓存数据。
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
**leijun** commented on *2022-12-23T16:22:04.442+0800*:
|
|||
|
|
|
|||
|
|
E21参考文档中步骤修复问题 https://docs.geedge.net/pages/viewpage.action?pageId=91324796
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
**zhangwei** commented on *2022-12-26T18:25:13.870+0800*:
|
|||
|
|
|
|||
|
|
清理历史版本的同步审计日志，观察是否可缓解同步程序读取数据库超时的情况。
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
**leijun** commented on *2022-12-26T18:47:52.665+0800*:
|
|||
|
|
|
|||
|
|
CM-Sync-Error告警查看error日志发现是由于数据库连接超时导致，通过排查keepalived日志未发现keepalived主从切换，bifang-api服务未发生数据库连接超时现象，最后经沟通确认发现t_rule_sync_history数据查询时间过长造成了数据库链接超时，由于bifang-sync服务频繁推送全量，从redis在大量同步数据，短暂连接不上造成CM_Branch_Redis告警
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
**liuju** commented on *2022-12-28T21:10:08.273+0800*:
|
|||
|
|
|
|||
|
|
目前已观察了一天，处理完毕之后未再出现以上告警消息。[~leijun]
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
## Attachments
|
|||
|
|
|
|||
|
|
**33885/alert-message-2022-12-22+15-47-46.xlsx**
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
**33884/微信图片_20221223094808.png**
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|