geedge-jira/md/OMPUB-723.md

# 福建项目：泉州移动4G功能端丢包

| ID | Creation Date | Assignee | Status |
|----|----------------|----------|--------|
| OMPUB-723 | 2022-12-02T20:18:48.000+0800 | 彭宣正 | 已关闭 |


---

192.168.13.1-4 4台功能端出现丢包现象
原因：服务器CPU爆满
但是近期无策略变化，目前观察分时段出现丢包现象**liuxueli** commented on *2022-12-02T20:48:23.773+0800*:

* 查看现场，CPU使用率较高
 ** 
{code:java}
%Cpu0  : 83.8 us,  2.4 sy,  0.0 ni, 13.5 id,  0.0 wa,  0.0 hi,  0.3 si,  0.0 st
%Cpu1  : 96.3 us,  3.0 sy,  0.0 ni,  0.7 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu2  : 95.0 us,  2.7 sy,  0.0 ni,  2.3 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu3  : 96.4 us,  1.7 sy,  0.0 ni,  2.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu4  : 93.4 us,  4.6 sy,  0.0 ni,  2.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu5  : 96.0 us,  2.3 sy,  0.0 ni,  1.7 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu6  : 89.7 us,  8.3 sy,  0.0 ni,  1.7 id,  0.0 wa,  0.0 hi,  0.3 si,  0.0 st
%Cpu7  : 93.3 us,  3.0 sy,  0.0 ni,  3.7 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu8  : 92.0 us,  3.7 sy,  0.0 ni,  4.3 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu9  : 95.7 us,  2.7 sy,  0.0 ni,  1.7 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu10 : 95.0 us,  3.0 sy,  0.0 ni,  2.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu11 : 94.0 us,  3.3 sy,  0.0 ni,  2.6 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu12 : 94.4 us,  2.3 sy,  0.0 ni,  3.3 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu13 : 77.6 us,  3.0 sy,  0.0 ni, 19.1 id,  0.0 wa,  0.0 hi,  0.3 si,  0.0 st
%Cpu14 : 81.3 us,  4.0 sy,  0.0 ni, 14.7 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu15 : 82.8 us,  2.0 sy,  0.0 ni, 14.8 id,  0.0 wa,  0.0 hi,  0.3 si,  0.0 st
%Cpu16 : 94.4 us,  2.0 sy,  0.0 ni,  3.6 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu17 : 95.4 us,  1.7 sy,  0.0 ni,  3.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu18 : 94.0 us,  1.7 sy,  0.0 ni,  4.3 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu19 : 88.7 us,  2.3 sy,  0.0 ni,  9.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu20 : 86.3 us,  3.0 sy,  0.0 ni, 10.7 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu21 : 93.4 us,  1.7 sy,  0.0 ni,  5.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu22 : 91.2 us,  2.0 sy,  0.0 ni,  6.7 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu23 : 83.6 us,  3.0 sy,  0.0 ni, 13.4 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu24 : 88.1 us,  2.3 sy,  0.0 ni,  9.6 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu25 : 79.3 us,  4.8 sy,  0.0 ni, 16.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu26 : 84.1 us,  4.1 sy,  0.0 ni, 11.8 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu27 : 79.5 us,  4.1 sy,  0.0 ni, 16.4 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu28 : 86.9 us,  2.3 sy,  0.0 ni, 10.7 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu29 : 76.5 us,  1.3 sy,  0.0 ni, 22.2 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu30 : 77.7 us,  1.7 sy,  0.0 ni, 20.7 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu31 : 76.7 us,  1.4 sy,  0.0 ni, 22.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu32 : 90.0 us,  3.7 sy,  0.0 ni,  6.4 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu33 : 97.0 us,  1.7 sy,  0.0 ni,  1.3 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu34 : 94.7 us,  1.7 sy,  0.0 ni,  3.7 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu35 : 93.4 us,  4.0 sy,  0.0 ni,  2.3 id,  0.0 wa,  0.0 hi,  0.3 si,  0.0 st
%Cpu36 : 94.7 us,  2.7 sy,  0.0 ni,  2.7 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu37 : 94.4 us,  2.3 sy,  0.0 ni,  3.3 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu38 : 94.4 us,  1.7 sy,  0.0 ni,  4.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu39 : 91.3 us,  4.7 sy,  0.0 ni,  4.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu40 : 93.7 us,  4.0 sy,  0.0 ni,  2.3 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu41 : 96.4 us,  2.0 sy,  0.0 ni,  1.3 id,  0.0 wa,  0.0 hi,  0.3 si,  0.0 st
%Cpu42 : 95.7 us,  2.0 sy,  0.0 ni,  2.0 id,  0.0 wa,  0.0 hi,  0.3 si,  0.0 st
%Cpu43 : 94.7 us,  2.3 sy,  0.0 ni,  2.6 id,  0.0 wa,  0.0 hi,  0.3 si,  0.0 st
 {code}

 * 使用perf top -C  cpuid(11/12)查看，multi_getsock和multi_runsingle函数占用CPU较多，如下图:
 ** !image-2022-12-02-20-45-15-083.png!
 ** !image-2022-12-02-20-46-03-148.png!



---

**pengxuanzheng** commented on *2022-12-05T10:45:57.154+0800*:

hos 修改配置 
hos_curl_easy_handle_per_multi_handle = 20， 降低curl_multi_perform调用频率



---

**pengxuanzheng** commented on *2022-12-06T11:33:40.186+0800*:

hos修改配置后，服务器现状：

!image-2022-12-06-11-36-25-098.png!

查看8,9,10,11 核，hos CPU使用率均有所下降

!image-2022-12-06-11-36-02-699.png!

观察一段时间发现，hos CPU使用率在2%-9%之间，可以加大hos_curl_easy_handle_per_multi_handle 至 50，继续降低curl_multi_perform调用频率



---

**pengxuanzheng** commented on *2022-12-06T14:28:18.014+0800*:

改完之后，功能端不再丢包



---

**pengxuanzheng** commented on *2022-12-06T14:43:54.475+0800*:

hos_curl_easy_handle_per_multi_handle 的作用是每个curl multi handle 收到多少个 curl easy handle 之后才会开始curl_multi_perform, 也就是一次curl_multi_perform 携带的put请求个数， 默认值是1（配合此前的交互式上传），后续会修改默认值为20



---



# Attachments

Attachment: image-2022-12-02-20-45-15-083.png

![image-2022-12-02-20-45-15-083.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/33344/image-2022-12-02-20-45-15-083.png)



Attachment: image-2022-12-02-20-46-03-148.png

![image-2022-12-02-20-46-03-148.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/33345/image-2022-12-02-20-46-03-148.png)



Attachment: image-2022-12-06-11-36-02-699.png

![image-2022-12-06-11-36-02-699.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/33397/image-2022-12-06-11-36-02-699.png)



Attachment: image-2022-12-06-11-36-25-098.png

![image-2022-12-06-11-36-25-098.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/33396/image-2022-12-06-11-36-25-098.png)