【E21现场】tsg_9140_packet_io_rxdrop告警

ID	Creation Date	Assignee	Status
OMPUB-609	2022-09-04T10:21:07.000+0800	刘学利	已关闭

目前有两块NPB丢包较为严重 ** Bole IGW 10.225.11.1，最长持续12小时丢包 *** !image-2022-09-05-09-45-43-357.png|width=653,height=316! ** LGH-PE 10.230.11.1，最长持续14小时丢包 *** !image-2022-09-05-09-48-21-227.png|width=656,height=315!liuxueli commented on 2022-09-29T10:44:32.666+0800:
新疆移动、福建、E现场对比 ** ||环境||机器||原始流量(Gbps)||平均包长(Byte)||CPU使用率|| |新疆|10.111.128.244|4.2|707|1000| |福建|192.168.13.5|25|1037|3900| |E现场|10.225.11.5|5.2|391|3800|

* ** 新疆采集日志 *** [^XJ-tsg_static_maat.txt] *** [^XJ-app_sketch_maat.txt]

{code:java} [root@cmcc-xj-server1-sapp-244 ~]# monit_device Time: Thu Sep 29 10:19:21 2022, Physical device: ens1f0 -------------- --------------- ------------------- ------------- ------------ ------------ ------------ ----------- ------------ ------------ ------------ PhyRXFrame PhyRXBits PhyRXMissed PhyRXError PhyRXNoBUF PhyTXFrame PhyTXBits PhyTXError UsrRXDrops UsrTXDrops -------------- --------------- ------------------- ------------- ------------ ------------ ------------ ----------- ------------ ------------ ------------ Accumulative 3402819273046 21578173059616072 300666 3 0 0 0 0 0 0 Per Second 362819 2094983120 0 0 0 0 0 0 0 0 -------------- --------------- ------------------- ------------- ------------ ------------ ------------ ----------- ------------ ------------ ------------ Time: Thu Sep 29 10:19:21 2022, Physical device: ens2f0 -------------- --------------- ------------------- ------------- ------------ ------------ ------------ ----------- ------------ ------------ ------------ PhyRXFrame PhyRXBits PhyRXMissed PhyRXError PhyRXNoBUF PhyTXFrame PhyTXBits PhyTXError UsrRXDrops UsrTXDrops -------------- --------------- ------------------- ------------- ------------ ------------ ------------ ----------- ------------ ------------ ------------ Accumulative 3386389018195 21447301997051296 323333 2 0 0 0 0 0 0 Per Second 350392 1947467376 0 0 0 0 0 0 0 0 -------------- --------------- ------------------- ------------- ------------ ------------ ------------ ----------- ------------ ------------ ------------ [root@cmcc-xj-server1-sapp-244 ~]# {code}

** 福建采集日志 *** [^FJ-tsg_static_maat.txt] *** [^FJ-app_sketch_maat.txt]

{code:java} [root@FJYD-QZ-CD5G-SERVER-5 ~]# monit_device Time: Thu Sep 29 10:27:44 2022, Physical device: enp94s0f0 -------------- --------------- ------------------- ------------- ------------ --------------- ------------ ----------- ------------ ------------ ------------ PhyRXFrame PhyRXBits PhyRXMissed PhyRXError PhyRXNoBUF PhyTXFrame PhyTXBits PhyTXError UsrRXDrops UsrTXDrops -------------- --------------- ------------------- ------------- ------------ --------------- ------------ ----------- ------------ ------------ ------------ Accumulative 5876088144744 49749311895205440 3850304909 0 1685329285504 0 0 0 0 0 Per Second 431568 3477451552 0 0 0 0 0 0 0 0 -------------- --------------- ------------------- ------------- ------------ --------------- ------------ ----------- ------------ ------------ ------------ Time: Thu Sep 29 10:27:44 2022, Physical device: enp94s0f1 -------------- --------------- ------------------- ------------- ------------ --------------- ------------ ----------- ------------ ------------ ------------ PhyRXFrame PhyRXBits PhyRXMissed PhyRXError PhyRXNoBUF PhyTXFrame PhyTXBits PhyTXError UsrRXDrops UsrTXDrops -------------- --------------- ------------------- ------------- ------------ --------------- ------------ ----------- ------------ ------------ ------------ Accumulative 5596233542162 46579937355105616 4232724156 2 1969018792544 0 0 0 0 0 Per Second 477823 4038176592 0 0 0 0 0 0 0 0 -------------- --------------- ------------------- ------------- ------------ --------------- ------------ ----------- ------------ ------------ ------------ Time: Thu Sep 29 10:27:44 2022, Physical device: enp94s0f2 -------------- --------------- ------------------- ------------- ------------ --------------- ------------ ----------- ------------ ------------ ------------ PhyRXFrame PhyRXBits PhyRXMissed PhyRXError PhyRXNoBUF PhyTXFrame PhyTXBits PhyTXError UsrRXDrops UsrTXDrops -------------- --------------- ------------------- ------------- ------------ --------------- ------------ ----------- ------------ ------------ ------------ Accumulative 5705803115296 47669672212376112 189927954 2 2155727294208 0 0 0 0 0 Per Second 508102 4218404072 0 0 0 0 0 0 0 0 -------------- --------------- ------------------- ------------- ------------ --------------- ------------ ----------- ------------ ------------ ------------ Time: Thu Sep 29 10:27:44 2022, Physical device: enp175s0f0 -------------- --------------- ------------------- ------------- ------------ --------------- ------------ ----------- ------------ ------------ ------------ PhyRXFrame PhyRXBits PhyRXMissed PhyRXError PhyRXNoBUF PhyTXFrame PhyTXBits PhyTXError UsrRXDrops UsrTXDrops -------------- --------------- ------------------- ------------- ------------ --------------- ------------ ----------- ------------ ------------ ------------ Accumulative 5576716259278 46436799420222776 3302927729 0 1659902871200 0 0 0 0 0 Per Second 576043 4935756616 0 0 0 0 0 0 0 0 -------------- --------------- ------------------- ------------- ------------ --------------- ------------ ----------- ------------ ------------ ------------ Time: Thu Sep 29 10:27:44 2022, Physical device: enp175s0f1 -------------- --------------- ------------------- ------------- ------------ --------------- ------------ ----------- ------------ ------------ ------------ PhyRXFrame PhyRXBits PhyRXMissed PhyRXError PhyRXNoBUF PhyTXFrame PhyTXBits PhyTXError UsrRXDrops UsrTXDrops -------------- --------------- ------------------- ------------- ------------ --------------- ------------ ----------- ------------ ------------ ------------ Accumulative 5708388358113 47883338536611896 3638379241 0 2023024473376 0 0 0 0 0 Per Second 482754 3949455696 0 0 0 0 0 0 0 0 -------------- --------------- ------------------- ------------- ------------ --------------- ------------ ----------- ------------ ------------ ------------ Time: Thu Sep 29 10:27:44 2022, Physical device: enp175s0f2 -------------- --------------- ------------------- ------------- ------------ --------------- ------------ ----------- ------------ ------------ ------------ PhyRXFrame PhyRXBits PhyRXMissed PhyRXError PhyRXNoBUF PhyTXFrame PhyTXBits PhyTXError UsrRXDrops UsrTXDrops -------------- --------------- ------------------- ------------- ------------ --------------- ------------ ----------- ------------ ------------ ------------ Accumulative 5874161898690 49630113644006880 151132015 0 2205368295584 0 0 0 0 0 Per Second 494508 4077464232 0 0 0 0 0 0 0 0 -------------- --------------- ------------------- ------------- ------------ --------------- ------------ ----------- ------------ ------------ ------------ [root@FJYD-QZ-CD5G-SERVER-5 ~]# {code}

** E现场采集日志 *** [^E21-tsg_static_maat.txt] *** [^E21-app_sketch_maat.txt]

{code:java} [admin@BOL-IGW-T9K001-NPB05 logcheck]$ monit_device Time: Wed Sep 28 16:24:37 2022, Physical device: eth_vf_raw -------------- --------------- ---------------------- ------------- ------------ ------------ --------------- ------------------ ------------ ------------ ------------ PhyRXFrame PhyRXBits PhyRXMissed PhyRXError PhyRXNoBUF PhyTXFrame PhyTXBits PhyTXError UsrRXDrops UsrTXDrops -------------- --------------- ---------------------- ------------- ------------ ------------ --------------- ------------------ ------------ ------------ ------------ Accumulative 4018026197029 18446691532562094080 324982 0 0 4017959299658 2199101554083744 0 0 0 Per Second 1812518 5681117368 0 0 0 1812633 5681916880 0 0 0 -------------- --------------- ---------------------- ------------- ------------ ------------ --------------- ------------------ ------------ ------------ ------------ Time: Wed Sep 28 16:24:37 2022, Physical device: eth_vf_kni -------------- ------------ ------------ ------------- ------------ ------------ ------------ ------------ ------------ ------------ ------------ PhyRXFrame PhyRXBits PhyRXMissed PhyRXError PhyRXNoBUF PhyTXFrame PhyTXBits PhyTXError UsrRXDrops UsrTXDrops -------------- ------------ ------------ ------------- ------------ ------------ ------------ ------------ ------------ ------------ ------------ Accumulative 527326 4934495784 0 0 0 402251 4536046816 0 0 0 Per Second 0 0 0 0 0 0 0 0 0 0 -------------- ------------ ------------ ------------- ------------ ------------ ------------ ------------ ------------ ------------ ------------ Time: Wed Sep 28 16:24:37 2022, Physical device: eth_vf_mirr -------------- ------------ ----------- ------------- ------------ ------------ ------------ ---------------- ------------ ------------ ------------ PhyRXFrame PhyRXBits PhyRXMissed PhyRXError PhyRXNoBUF PhyTXFrame PhyTXBits PhyTXError UsrRXDrops UsrTXDrops -------------- ------------ ----------- ------------- ------------ ------------ ------------ ---------------- ------------ ------------ ------------ Accumulative 0 0 0 0 0 4372605912 14063770621416 0 0 0 Per Second 0 0 0 0 0 1792 2074648 0 0 0 -------------- ------------ ----------- ------------- ------------ ------------ ------------ ---------------- ------------ ------------ ------------ {code}

yangwei commented on 2023-07-03T11:57:19.229+0800:

[~songlongkun] 麻烦补充下现场对应NPB最新的监控情况，如未复现，考虑先关闭本issue。

heyong commented on 2023-07-03T14:30:19.809+0800:

近期现场未出现持续时间超过3m的丢包情况，关闭本issue并将现场丢包情况观察及监控合并至https://jira.geedge.net/browse/OMPUB-934

继续进行观察。