geedge-jira/md/OMPUB-466.md

# 【E21现场】IGW站点security policy  block  www.target.com 失败

| ID | Creation Date | Assignee | Status |
|----|----------------|----------|--------|
| OMPUB-466 | 2022-04-26T22:02:07.000+0800 | 刘学利 | 已关闭 |


---

业主自己尝试配置security policy  block 网站：[https://www.target.com/] ，

策略配置如下：

client ip=196.188.136.150（办公室公网ip）

application：ssl 

sni-FQDN:*target.com

                 *targetimg1.com

                  $target.scene7.com

                  $target.com

Sub Action:reset

Effective Devices：{color:#0747a6}*所有IGW站点*{color}

 

策略测试效果：

浏览器一直访问[https://www.target.com/ |https://www.target.com/]，最初无法访问成功，大概一分钟左右网站可以正常访问。

 

并将以下内容上传到附件中：

策略配置内容截图

处理机计算板上drop情况

该策略安全策略命中日志

会话日志里搜索sni =%target% client ip=196.188.136.150导出session records

 

查询安全策略命中日志和会话日志时stream direction 只看到double 和c2s,没有s2c日志记录。

 

 **liuxueli** commented on *2022-04-27T10:01:47.937+0800*:

* [~liuju] 这种问题需要现场捕包，在客户端捕包发我，我分析一下
 * 另外我在京版尝试能不能复现



---

**dongxiaoyan** commented on *2022-04-27T10:44:19.639+0800*:

[~liuxueli]信息港复测：
1、浏览器正常访问，多访问几次打开一次；
2、无痕模式访问，一直未打开；
3、再次清理缓存后，浏览器正常访问，隔几分钟刷新一次，一直未打开；



---

**liuxueli** commented on *2022-04-27T11:45:50.356+0800*:

* 京版复现情况(京版环境不能稳定复现)：

 * 
 ** 京版唯一复现的一次，同时开启了捕包，数据包中存在18个ssl链接的SNI包含(target.com)的域名，
 *** 16个链接在client hello收到功能端发出的RESET后，链接关闭
 *** 2个链接未收到RESET,，链接正常建立
 **** 在SessionRecord中未查到该2个链接的记录，客户端端口为17375/17376
 **** SSL解析层测试18个链接均能解析出SNI
 *** [^18-target.com.pcap]



---

**gitlab** commented on *2022-04-27T11:49:29.384+0800*:

[刘学利|https://git.mesalab.cn/liuxueli] mentioned this issue in [a commit|https://git.mesalab.cn/MESA_Platform/ssl/-/commit/8e49d1f437fe377a6dabae5bcbc623c2bc34b5d3] of [MESA Platform / ssl|https://git.mesalab.cn/MESA_Platform/ssl] on branch [bugfix-add-test-case|https://git.mesalab.cn/MESA_Platform/ssl/-/tree/bugfix-add-test-case]:{quote}OMPUB-466: 增加相应的测试用例{quote}



---

**liuxueli** commented on *2022-04-27T11:51:39.774+0800*:

* [bugfix-add-test-case|https://git.mesalab.cn/MESA_Platform/ssl/-/tree/bugfix-add-test-case]: 分支未修复任何BUG，仅增加测试用例，需继续定位该问题。



---

**liuxueli** commented on *2022-04-27T15:20:25.195+0800*:

* E现场复测，客户端未收到RESET包，查看日志发现RESET包发送失败报错
 ** 参见：TSG-10508 



---

**liuxueli** commented on *2022-04-27T17:29:09.456+0800*:

* 分析E现场反馈回来数据包，有3个链接未阻断成功，经分析原因：在Bole IGW站点的CPU使用过载触发SAPP overload protection机制导致链接被Bypass(sapp未建立对应的流表)
 ** 当单核CPU使用率超95%时，会触发SAPP的overload protection机制，SAPP配置文件如下：

 * 
 ** 
 *** !image-2022-04-27-17-53-42-859.png!  
 ** 分析在NPB上捕获的数据包，找到3个未阻断成功链接对应的client ISN，查询SessionRecord日志，仅在Old Airport站点查到日志
 *** NPB捕获的数据包
 **** [^target.com-196.188.136.150-151.101.2.187.443.pcap] 
 *** Session日志记录
 **** [^sessionRecords-deny-target.com-failed.xlsx]
 ** 查看NPB DDOS Bypass监控，TCP Bypass 3~5K/S个链接，UDP Bypass 1.55K/S个链接：
 *** !image-2022-04-27-17-12-00-094.png|width=1090,height=508!
 ** 查看NPB原始流量
 *** !image-2022-04-27-17-14-25-901.png|width=1093,height=506!
 **  查看NPB 整体CPU使用率
 *** !image-2022-04-27-17-15-02-774.png|width=1079,height=492!
 ** 查看NPB 单核CPU使用率
 *** !image-2022-04-27-17-17-32-639.png!
 ** 查看perf top -C cpuid的结果
 *** !image-2022-04-27-17-19-09-881.png!  
 *** !image-2022-04-27-17-18-43-458.png!
 ** 查看火焰图
 *** [^perf.svg]
 *** [^perf33.svg]
 *** [^perf37.svg]
 *** [^perf40.svg]



---

**liuxueli** commented on *2022-04-27T17:36:10.537+0800*:

* 火焰图中_int_malloc调用较多的地方
 ** tsg_master
 *** !image-2022-04-27-17-36-03-049.png|width=1139,height=433!
 ** MAAT
 *** !image-2022-04-27-17-35-20-050.png|width=1152,height=213!
 **  



---

**liuxueli** commented on *2022-04-28T10:27:29.680+0800*:

* E现场反馈回来的扫描状态显示，FQDN命中率较高：
 ** APP SKETCH扫描状态[^app_sketch_maat.txt]
 *** FQDN识别特征命中率为80%，
 *** APP_SIG_SESSION_ATTRIBUTE_STRING表命中率: 35%
 *** APP_SIG_SESSION_ATTRIBUTE_INTEGER表命中率: 10%
 ** TSG功能端扫描状态[^tsg_static_maat.txt]
 *** FQDN命中率为: 58%
 *** KEYWORDS命中率: 3.6%
 *** APP_ID命中率: 96%(无性能影响)
 *** FQDN_CAT命中率: 100%(无性能影响)



---

**liuxueli** commented on *2022-04-28T11:03:48.107+0800*:

* [~liuju] 到现场后备份一下Bifang的MariaDB，我需要验证现场的哪个FQDN的对象命中率比较高
 ** 京版环境可能跟现场有差别
 * E现场备份回来的21.11版本，存在重复的FQDN Object现象，具体列表：
 ** [^e21-version-21.11-dup-fqdn-object.txt]
 ** ^经分析确认为APP内置特征(VPN)，一个FQDN在多个Object中出现^
 ** ^对应BUG: TSG-10517^



---

**liuxueli** commented on *2022-04-28T14:34:33.918+0800*:

* E现场备份回来的Bifang MariaDB v21.11版本发现，APP内置特征中包含三个极易命中的特征
 ** APP_SIG_SESSION_ATTRIBUTE_STRING
 *** 
{code:java}
12482085    113748    http.user_agent    chrome    0    0    0    1    1639035425000000 0
9021592    112096    http.user_agent    Chrome    0    0    0    1    1637987142000000 0
9021593    112097    http.content_type    text/html    0    0    0    1    1637987142000000 0 {code}



---

**zhengchao** commented on *2022-04-28T14:41:38.582+0800*:

现场http的比例多大？是触发overload protection的原因吗？



---

**liuxueli** commented on *2022-04-28T15:43:28.368+0800*:

* Bole IGW NPB5的协议统计，HTTP的比例8%，可能是APP_SIG_SESSION_ATTRIBUTE_STRING表命中率较高的原因
 ** 导出PDF结果中QUIC拼写错误，参见：TSG-10523  
 ** [^L7protocal_sessions-Bole IGW NPB2.csv]
 ** [^L7protocal_sessions-Bole IGW NPB5.csv]
 **  [^L7protocal_sessions-all-NPB.csv]
 ** [^L7protocal_sessions-all-NPB-new.csv]
 ** !image-2022-04-28-15-43-09-227.png!



---

**liuxueli** commented on *2022-04-28T16:27:44.265+0800*:

* FQDN扫描命中率高的原因应该是：APP_SKETCH扫描了TOPN SNI，识别Psiphon3和Freegate非的条件。
 ** [^Bole IGW NPB5-http_domain_session.csv]
 ** [^Bole IGW NPB5-QUIC.SNI_session.csv]
 ** [^Bole IGW NPB5-SSL.SNI_session.csv]



---

**liuxueli** commented on *2022-04-28T16:54:53.829+0800*:

* APP_SKETCH扫描了TOPN SNI高命中率导致的CPU跑满，从而引发Bole IGW站点的SAPP overload protection机制导致链接被Bypass。[~luqiuwen] 



---

**zhengchao** commented on *2022-04-28T17:01:59.390+0800*:

IGW能不能把TFE的CPU让出来？[~luqiuwen] 



---

**luqiuwen** commented on *2022-04-28T17:02:57.016+0800*:

可以在IGW的计算板上关闭Proxy功能，将Proxy使用的CPU资源调整为Firewall使用。Provision中已预留相关选项，操作步骤：
 * 修改/data/tsg-os-provision/provision.yml文件，将proxy->enable选项由1调整为0：

{code:java}
proxy:
  enable: 1 {code}
修改为：
{code:java}
proxy:
  enable: 0 {code}
 * 令上述配置生效，运行：

{code:java}
$ sudo provision-config-apply{code}
 * 检查tfe是否在运行

{code:java}
$ systemctl status tfe
● tfe.service - Tango Frontend Engine
   Loaded: loaded (/usr/lib/systemd/system/tfe.service; disabled; vendor preset: disabled)
  Drop-In: /usr/lib/systemd/system/tfe.service.d
           └─require-mrzcpd.conf, service_add_ConditionPathExists.conf, service_override_env.conf, service_override_slice.conf
   Active: inactive (dead) since Thu 2022-04-28 11:46:17 EAT; 15min ago
  Process: 18732 ExecStart=/opt/tsg/tfe/bin/tfe (code=killed, signal=TERM)
 Main PID: 18732 (code=killed, signal=TERM){code}
其中，Active项应为inactive。

需要考虑的问题是：
 * 在关闭Proxy功能后，所有拦截策略和代理策略将无法在IGW生效，如何向业主解释。



---

**zhengchao** commented on *2022-04-28T17:06:42.366+0800*:

对业主解释：由于IGW存在单向流，Proxy功能不在IGW生效。



---

**gitlab** commented on *2022-04-28T18:18:44.475+0800*:

[刘学利|https://git.mesalab.cn/liuxueli] mentioned this issue in [a merge request|https://git.mesalab.cn/MESA_Platform/ssl/-/merge_requests/34] of [MESA Platform / ssl|https://git.mesalab.cn/MESA_Platform/ssl] on branch [feature-add-test-case|https://git.mesalab.cn/MESA_Platform/ssl/-/tree/feature-add-test-case]:{quote}OMPUB-466: 增加相应的测试用例{quote}



---

**liuxueli** commented on *2022-04-29T09:44:47.117+0800*:

* Bole IGW NPB5 TFE的CPU让出来后，被SAPP主动Bypass的链接有减少，但是没有完全消除
 ** !image-2022-04-29-09-44-38-025.png|width=1108,height=549!



---

**liuxueli** commented on *2022-04-29T10:19:39.687+0800*:

* [~liuju] 到现场后把/opt/tsg/sapp/sysinfo.log文件发回来一份。
 ** [^Bole IGW NPB5 sysinfo.log]



---

**liuxueli** commented on *2022-05-05T14:23:24.256+0800*:

* 新疆环境统计，xx.xxx.192.175：
 ** 实时流量约6Gbps，monit_device统计：
 ***  
 *** !XJ-192.175-monit_device.png!
 ** 整体的CPU使用率，单核使用率均在60~65%左右：
 *** !XJ-192.175-top-all-cpu.png|thumbnail!
 ** SAPP的TCP/UDP链接流标大小为100000，sysinfo统计： [^XJ-192.175-sysinfo.txt]
 *** !XJ-192.175-sysinfo.png!
 ** TSG MAAT句柄的扫描状态统计：[^XJ-192.175-tsg_static_maat.txt]
 *** FQDN字符串扫描命中率为：53%
 *** IP归属地字符串扫描命中率为：51% 
 *** !XJ-192.175-tsg-maat.png!
 ** APP MAAT句柄扫描统计：[^XJ-192.175-app_sketch_maat.txt]
 ***  FQDN字符串扫描命中率为：66%
 *** !XJ-192.175-app-maat.png!
 ** 单核CPU使用，perf top -C 6：
 *** !XJ-192.175-perf-top-cpu6.png!
 ** CPU 6的火焰图[^XJ-192.175-perf-cpu-6.svg]
 *** ^__clock_gettime 是SAPP统计包处理延时造成的，E现场未开启该功能^



---

**zhengchao** commented on *2022-05-05T14:45:02.941+0800*:

FQDN对比，命中率、扫描性能({_}PROC_Tps{_})相近，E21的扫描次数{_}(IN_Tps=5.30e+04){_}是新疆(1.62e+04)的3倍
{code:java}
TSG_OBJ_FQDN (XJ)                160158             0             0      2.86e+05      5.66e+06      1.62e+04      3.20e+05      5.72e-01
TSG_OBJ_FQDN (E21)                 4532             0             0      1.15e+06      7.26e+06      5.30e+04      3.36e+05      5.80e-01 {code}
 



---

**yangwei** commented on *2022-05-05T14:53:03.447+0800*:

补充说明：
 # 新疆单核perf排名第一的函数为clock_get_time，原因是sapp开启了包处理延迟统计，E现场没有开启该功能
 # 新疆CPU型号为Intel(R) Xeon(R) Silver 4114 CPU @ 2.20GHz，9140的CPU型号为Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz



---

**liuxueli** commented on *2022-05-06T11:00:08.046+0800*:

* 使用京版E现场升级验证环境测试，仪表构造流量，加载E现场的配置：
 ** {color:#de350b}测试结果：不加载E现场的APP特征配置，新建性能可以提升一倍{color}
 *** {color:#de350b}新建由3.3万/s提升到7.3wan/s{color}
 *** {color:#de350b}其中加载E现场APP特征，损耗CPU较多的函数为“msort_with_tmp”{color}
 ** {color:#172b4d}测试分为两组对比测试，{color}

 * 
 ** 
 *** {color:#172b4d}加载E现场APP特征配置{color}
 **** {color:#172b4d}整体CPU使用率：{color}
 ***** {color:#172b4d}!XXG-40.82-load-app-maat-all-cpu.png!{color}
 **** {color:#172b4d}单核CPU使用率，perf top -C 14{color}
 ***** {color:#172b4d}!XXG-40.82-load-app-maat-perf-top-cpu14.png!{color}
 **** SAPP处理新建连接数({color:#de350b}3.3万/s{color})：[^XXG-40.82-load-app-maat-sysinfo.txt]
 ***** !XXG-40.82-load-app-maat-new-link.png!
 *** {color:#172b4d}不加载E现场APP特征配置{color}
 **** {color:#172b4d}整体CPU使用率：{color}
 ***** !XXG-40.82-no-app-maat--all-cpu.png!
 **** {color:#172b4d}单核CPU使用率，perf top -C 14{color}
 ***** {color:#172b4d}!XXG-40.82-no-app-maat-perf-top-cpu14.png!{color}
 **** {color:#172b4d}SAPP处理新建连接数(7.3万/s{color})，[^XXG-40.82-no-app-maat-sysinfo.txt]：
 ***** !XXG-40.82-no-app-maat-new-link.png!



---

**liuxueli** commented on *2022-05-06T11:01:15.064+0800*:

* 等待[~zhangwei] 优化APP特征组织结构后进行复测。
 ** 去除APP特征中APP ID的引用



---

**yangwei** commented on *2022-05-06T18:21:12.898+0800*:

分析E现场使用的App特征，在APP_SIG_SESSION_ATTRIBUTE_INTEGER表中，general.session.analysis.app_id存在较多重复
 * 如下图所示，第一列为重复次数，第二列为AppID，其中67为HTTP，199为SSL，按前述统计，这俩种协议分别占E现场流量的65%和8%

可以部分解释前述在新建连接较高的情况下，单核出现CPU使用较高（perf采样占用第一的为msort_xxx函数调用）导致触发sapp自我保护的现象

!image-2022-05-06-18-15-08-649.png!



---

**liuxueli** commented on *2022-05-07T16:16:36.389+0800*:

* 优化APP特征组织结构后进行复测（去除APP特征中APP ID的引用），使用同一个数据包进行测试，对比APP的识别结果
 ** {color:#de350b}优化前单核新建处理能力为: 1000/s，单核CPU使用率75%左右{color}
 ** {color:#de350b}优化后单核新建处理能力为: 2500/s，单核CPU使用率80%左右 {color}
 ** {color:#de350b}有3个APP识别结果不一致{color}
 *** {color:#de350b}!image-2022-05-07-16-12-05-474.png!{color}



---

**zhengchao** commented on *2022-05-07T17:00:00.363+0800*:

确认优化后的AppSketchDB可以在22.02版本正确加载，之后提供给E21现场。



---

**liuxueli** commented on *2022-05-09T10:51:10.555+0800*:

* App_GooseVPN_20211208识别结果存在差别，分析App_GooseVPN_20211208特征，特征条件变化，可能导致一定的误识别
 ** {color:#de350b}优化前特证：IP+PROTOCOl(IKE){color}
 ** {color:#de350b}优化后特征：仅IP{color}



---

**liuxueli** commented on *2022-05-09T17:24:52.318+0800*:

*  调整App_GooseVPN_20211208的特征，引用app_id=IKE的特征保留，
 * 手动删除其余对app_id特征的引用，优化前后的APP识别结果一致（Psiphon3、unknown除外）。
 ** {color:#de350b}E现场升级验证环境，Psiphon3的识别特征有变化，导致识别结果存在差异，识别结果变少{color}
 ** {color:#de350b}Psiphon3识别结果减少，unknown相应会增加{color}
 * {color:#172b4d}已提供至[~liuju] E21现场更新{color}



---

**liuju** commented on *2022-05-09T22:01:03.929+0800*:

1、已根据张东旭提供的《E现场修正app特征操作说明》文档，完成对E现场列表里提供的所有app自定义特征进行修改更新，将general.session.analysis.app_id的Condition删除。

2、并根据提供的sql语句对更新结果进行验证，验证更新无误。[~liuxueli] 



---

**liuxueli** commented on *2022-05-10T09:37:05.586+0800*:

* [~liuju] 观察Nezha监控上的DDOS Bypass的统计是否还存在？
 * 统计自定义APP的特征修正前后识别链接数是否较大的差异（统计12小时）？
 ** 统计Session Record日志中的common_app_label字段
 * 登录Bole  IGW NPB5查看一下整体CPU，使用perf top -C cpuid看一下单核的CPU



---

**liuxueli** commented on *2022-05-10T17:47:36.817+0800*:

* 建议[~liuju] 申请在IGW站点复测业主提出的网站阻断效果不佳的问题
 * 优化APP自定义特征后，观察Bole IGW的统计，{color:#de350b}其中由于SAPP overload protection机制导致链接被Bypass的统计(DDOS Bypass)降为0{color}，{color:#de350b}偶尔会存在几个链接被Byapss{color}
 ** Nezha统计新建及DDOS Bypass
 *** !Bole IGW  NPB1 new connect.png!
 *** !Bole IGW  NPB1 DDOS Bypass  connect.png!
 *** !Bole IGW  NPB2 new connect.png!
 *** !Bole IGW  NPB2 DDOS Bypass  connect.png!
 *** !Bole IGW  NPB3 new connect.png!
 *** !Bole IGW  NPB3 DDOS Bypass  connect.png!



---

**liuxueli** commented on *2022-05-10T17:50:27.077+0800*:

* [~liuju] Bole IGW NPB5 Nezha采集出现异常，请另外提一个BUG。



---

**liuju** commented on *2022-05-10T21:19:49.593+0800*:

[~liuxueli] 好的 采集问题已提交BUG  另外在IGW站点业主提出的网站阻断效果不佳的问题，业主已在进行复测，后续有结果会继续反馈



---

**liuxueli** commented on *2022-05-13T17:39:22.275+0800*:

* 从现场Nezha监控看，{color:#de350b}目前Bole IGW NPB5还存在由于SAPP overload protection机制导致链接被Bypass，流量峰值(10Gbps左右)时Bypass 2K/s个链接，CPU使用率在60%左右{color}
 ** {color:#172b4d}OMPUB-481影响，20220511~20220512统计出现异常，在20220512北京时间晚上重启sapp恢复统计{color}
 ** {color:#172b4d}DDOS Bypass统计{color}
 *** {color:#172b4d}!Bole IGW NPB5 DDOS bypass 20220513.png|width=1743,height=810!{color}
 ** {color:#172b4d}新建链接统计{color}
 *** {color:#172b4d}!Bole IGW NPB5 new connections 20220513.png|width=1777,height=827!{color}
 ** {color:#172b4d}流量统计{color}
 *** {color:#172b4d}!Bole IGW NPB5 throughtput 20220513.png|width=1581,height=743!{color}
 ** {color:#172b4d}CPU使用率{color}
 *** !Bole IGW NPB5 CPU 20220513.png!
 ** {color:#172b4d}perf top -C 32/44{color}
 *** {color:#172b4d}!Bole IGW NPB5 perf top cpu32 20220513.png!{color}
 *** {color:#172b4d}!Bole IGW NPB5 perf top cpu44 20220513.png!{color}



---

**liuxueli** commented on *2022-05-16T15:33:32.233+0800*:

* [~liuju] 建议关闭Bole IGW NPB SAPP的bypass功能，修改配置项：
 ** 修改/data/tsg-os-provision/provision.yml文件，将feature->enable_stream_bypass_under_ddos选项由1调整为0
 *** !image-2022-05-16-15-33-27-227.png!
 ** 执行命令使上述配置生效，运行：
 *** sudo provision-config-apply
 ** 执行完成后检查sapp配置文件etc/sapp.toml
 *** stream_bypass_enabled=0 
 *** !image-2022-05-16-15-32-36-510.png!



---

**liuju** commented on *2022-05-16T23:59:04.497+0800*:

BOLE-IGW 10.225.11.1~5 已经按你的更新要求完成更新现在，均已检查完更新之后配置stream_bypass_enabled=0。[~liuxueli] 待明天申请配置策略效果验证完，告诉更新之后的效果。



---

**liuxueli** commented on *2022-05-17T08:50:35.226+0800*:

* [~liuju] 观察Bole IGW 是否存在丢包现象。



---

**liuju** commented on *2022-05-17T20:50:49.691+0800*:

关闭Bole IGW NPB SAPP的bypass功能后，今天业主重新进行复测IGW站点 security policy deny功能，复测结果都deny成功。[~liuxueli] 墨处询问该问题的具体原因，我回复说我需要告诉你们复测结果之后，待家里你们再告知我具体问题原因。墨处要求明天知道问题原因。



---

**zhengchao** commented on *2022-05-17T21:09:19.754+0800*:

回复业主：问题原因北京还在定位。

[~liuju]  观察Bole IGW 是否存在丢包现象。



---

**liuju** commented on *2022-05-17T21:14:30.862+0800*:

收到，好的[~zhengchao]   BOLE-IGW 早上和现场观察结果还存在丢包。具体每个NPB丢包情况已微信提供给学利。



---

**liuxueli** commented on *2022-05-18T10:04:24.965+0800*:

* Bole IGW 存在偶尔丢包的情况，范围在200~1300pps，[~liuju] 需要同步丢包时刻的流量、CPU的监控
 ** !Bole IGW  NPB1 application drop pkts.png!
 ** !Bole IGW  NPB2 application drop pkts.png!
 ** !Bole IGW  NPB3 application drop pkts.png!
 ** !Bole IGW  NPB4 application drop pkts.png!
 ** !Bole IGW  NPB5 application drop pkts.png!



---

**liuju** commented on *2022-05-18T15:18:25.026+0800*:

好的[~liuxueli]  BOLE-IGW 最近24小时丢包及流量、CPU情况 截图 因本地网络上传jira困难，已将数据图片微信发送给你。



---

**liuxueli** commented on *2022-05-24T17:07:53.129+0800*:

* [~liuju] 观察Nezha监控，找一个由于CPU使用过载触发SAPP overload protection机制导致链接被Bypass较多得NPB，调整SAPP的配置文件，观察Bypass的情况
 ** 调整SAPP参数，位于sapp.toml，{color:#de350b}bypass_trigger_cpu_usage参数值由90改为99{color}，重启SAPP，观察bypass的情况
 *** !image-2022-05-24-17-06-33-618.png!



---

**liuju** commented on *2022-05-25T15:15:02.793+0800*:

[~zhengchao] 超哥，业主处长这近期一直追问进展，故障原因。



---

**liuju** commented on *2022-05-25T15:31:32.750+0800*:

[~liuxueli] 已对比近7天bypass数据之后 挑选了MWV-IGW 10.227.11.9  修改了/opt/tsg/sapp/etc/sapp.toml配置内容bypass_trigger_cpu_usage=90改完bypass_trigger_cpu_usage=99,已重启sapp，待观察更新后效果。



---

**zhengchao** commented on *2022-05-25T16:56:30.458+0800*:

[~liuxueli] 提供新的TSG OS文件，22.02继续升级。

IGW站点的overload bypass的阈值调整为99。[~yangwei] 

请 [~liuju] 提供现场perf，研发进一步优化性能。



---

**zhengchao** commented on *2022-05-25T16:58:01.728+0800*:

回复业主：由于之前的App特征过于消耗计算资源，影响了阻断功能，目前已优化了一轮，正在线上观察效果。北京方面也在开展更进一步的优化。
{quote}超哥，业主处长这近期一直追问进展，故障原因。
{quote}



---

**liuju** commented on *2022-05-25T22:09:07.905+0800*:

嗯嗯，好的~收到！



---

**liuxueli** commented on *2022-05-26T15:24:58.130+0800*:

* 20220526查看Bole IGW NPB5的CPU使用情况，查看perf top cpu48的使用分布
 ** 使用top查看整体CPU使用
 *** !Bole IGW NPB5 all cpu 20220526.png!
 ** perf top cpu48结果
 *** !Bole IGW NPB5 perf top cpu48 20220526.png!



---

**liuxueli** commented on *2022-05-31T09:46:06.827+0800*:

* [~liuju] 鉴于IGW站点的overload bypass的阈值调整为99后还存在Bypass的现象，找两个Bypass较多的NPB
 ** 一个NPB使用cpuages采集CPU的使用率，采集24小时
 ** 一个NPB调整sapp配置文件(/opt/tsg/sapp/etc/sapp.toml)参数，smooth_avg_window有2调整为20
 *** !image-2022-05-31-09-44-22-020.png!
 ** !image-2022-05-31-09-45-23-952.png!
 ** !image-2022-05-31-09-46-02-588.png!



---

**liuju** commented on *2022-05-31T14:28:57.759+0800*:

收到  好的[~liuxueli] 



---

**liuxueli** commented on *2022-06-10T15:38:27.387+0800*:

* NPB调整sapp配置文件(/opt/tsg/sapp/etc/sapp.toml)参数，smooth_avg_window有2调整为20，还是存在被SAPP Bypass的链接
 ** !image-2022-06-10-15-36-23-768.png!
 ** !image-2022-06-10-15-37-33-666.png!
 ** !image-2022-06-10-15-38-10-175.png!
 **  



---

**liuxueli** commented on *2022-06-10T16:49:20.613+0800*:

* [~liuju] smooth_avg_window参数由2调整为20 的NPB，使用cpusage命令采集一下CPU的使用率



---

**liuxueli** commented on *2022-06-15T10:14:53.829+0800*:

* smooth_avg_window参数由2调整为20 的NPB，还是存在Bypass的情况，查看cpusage采集的CPU使用率，Bypass前后时刻的CPU单核使用率均未超过95%，[~yangwei] 
 ** 采集日志及截图存在于: 40.146:/home/E21/CPU20220610~20220611.zip



---

**liuxueli** commented on *2022-06-17T16:45:53.990+0800*:

* 京版9140环境也存在链接被SAPP Bypass的现象。
 ** !XXG-9140.sapp.bypass.png!



---

**gitlab** commented on *2022-07-29T10:24:05.585+0800*:

[刘学利|https://git.mesalab.cn/liuxueli] mentioned this issue in [a merge request|https://git.mesalab.cn/MESA_Platform/ssl/-/merge_requests/35] of [MESA Platform / ssl|https://git.mesalab.cn/MESA_Platform/ssl] on branch [feature-add-test-case|https://git.mesalab.cn/MESA_Platform/ssl/-/tree/feature-add-test-case]:{quote}OMPUB-466: 增加SSL相应的测试用例{quote}



---

**gitlab** commented on *2022-07-29T10:24:09.209+0800*:

[刘学利|https://git.mesalab.cn/liuxueli] mentioned this issue in [a commit|https://git.mesalab.cn/MESA_Platform/ssl/-/commit/1325788848f31f58d173c57767a2d1d47a0c2598] of [MESA Platform / ssl|https://git.mesalab.cn/MESA_Platform/ssl] on branch [master|https://git.mesalab.cn/MESA_Platform/ssl/-/tree/master]:{quote}OMPUB-466: 增加SSL相应的测试用例{quote}



---



# Attachments

Attachment: 18-target.com.pcap

[18-target.com.pcap](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27574/18-target.com.pcap)



Attachment: app_sketch_maat.txt

[app_sketch_maat.txt](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27641/app_sketch_maat.txt)



Attachment: Bole+IGW++NPB1+application+drop+pkts.png

![Bole+IGW++NPB1+application+drop+pkts.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/28062/Bole+IGW++NPB1+application+drop+pkts.png)



Attachment: Bole+IGW++NPB1+application+drop+pkts-1.png

![Bole+IGW++NPB1+application+drop+pkts-1.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/28067/Bole+IGW++NPB1+application+drop+pkts-1.png)



Attachment: Bole+IGW++NPB1+DDOS+Bypass++connect.png

![Bole+IGW++NPB1+DDOS+Bypass++connect.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27863/Bole+IGW++NPB1+DDOS+Bypass++connect.png)



Attachment: Bole+IGW++NPB1+new+connect.png

![Bole+IGW++NPB1+new+connect.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27862/Bole+IGW++NPB1+new+connect.png)



Attachment: Bole+IGW++NPB2+application+drop+pkts.png

![Bole+IGW++NPB2+application+drop+pkts.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/28063/Bole+IGW++NPB2+application+drop+pkts.png)



Attachment: Bole+IGW++NPB2+application+drop+pkts-1.png

![Bole+IGW++NPB2+application+drop+pkts-1.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/28068/Bole+IGW++NPB2+application+drop+pkts-1.png)



Attachment: Bole+IGW++NPB2+DDOS+Bypass++connect.png

![Bole+IGW++NPB2+DDOS+Bypass++connect.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27865/Bole+IGW++NPB2+DDOS+Bypass++connect.png)



Attachment: Bole+IGW++NPB2+new+connect.png

![Bole+IGW++NPB2+new+connect.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27864/Bole+IGW++NPB2+new+connect.png)



Attachment: Bole+IGW++NPB3+application+drop+pkts.png

![Bole+IGW++NPB3+application+drop+pkts.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/28064/Bole+IGW++NPB3+application+drop+pkts.png)



Attachment: Bole+IGW++NPB3+application+drop+pkts-1.png

![Bole+IGW++NPB3+application+drop+pkts-1.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/28069/Bole+IGW++NPB3+application+drop+pkts-1.png)



Attachment: Bole+IGW++NPB3+DDOS+Bypass++connect.png

![Bole+IGW++NPB3+DDOS+Bypass++connect.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27867/Bole+IGW++NPB3+DDOS+Bypass++connect.png)



Attachment: Bole+IGW++NPB3+new+connect.png

![Bole+IGW++NPB3+new+connect.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27866/Bole+IGW++NPB3+new+connect.png)



Attachment: Bole+IGW++NPB4+application+drop+pkts.png

![Bole+IGW++NPB4+application+drop+pkts.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/28065/Bole+IGW++NPB4+application+drop+pkts.png)



Attachment: Bole+IGW++NPB4+application+drop+pkts-1.png

![Bole+IGW++NPB4+application+drop+pkts-1.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/28070/Bole+IGW++NPB4+application+drop+pkts-1.png)



Attachment: Bole+IGW+NPB5+all+cpu+20220526.png

![Bole+IGW+NPB5+all+cpu+20220526.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/28268/Bole+IGW+NPB5+all+cpu+20220526.png)



Attachment: Bole+IGW++NPB5+application+drop+pkts.png

![Bole+IGW++NPB5+application+drop+pkts.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/28066/Bole+IGW++NPB5+application+drop+pkts.png)



Attachment: Bole+IGW++NPB5+application+drop+pkts-1.png

![Bole+IGW++NPB5+application+drop+pkts-1.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/28071/Bole+IGW++NPB5+application+drop+pkts-1.png)



Attachment: Bole+IGW+NPB5+CPU+20220513.png

![Bole+IGW+NPB5+CPU+20220513.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27980/Bole+IGW+NPB5+CPU+20220513.png)



Attachment: Bole+IGW+NPB5+DDOS+bypass+20220513.png

![Bole+IGW+NPB5+DDOS+bypass+20220513.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27978/Bole+IGW+NPB5+DDOS+bypass+20220513.png)



Attachment: Bole+IGW+NPB5-http_domain_session.csv

[Bole+IGW+NPB5-http_domain_session.csv](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27668/Bole+IGW+NPB5-http_domain_session.csv)



Attachment: Bole+IGW+NPB5+new+connections+20220513.png

![Bole+IGW+NPB5+new+connections+20220513.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27979/Bole+IGW+NPB5+new+connections+20220513.png)



Attachment: Bole+IGW+NPB5+perf+top+cpu32+20220513.png

![Bole+IGW+NPB5+perf+top+cpu32+20220513.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27976/Bole+IGW+NPB5+perf+top+cpu32+20220513.png)



Attachment: Bole+IGW+NPB5+perf+top+cpu44+20220513.png

![Bole+IGW+NPB5+perf+top+cpu44+20220513.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27975/Bole+IGW+NPB5+perf+top+cpu44+20220513.png)



Attachment: Bole+IGW+NPB5+perf+top+cpu44+20220513-1.png

![Bole+IGW+NPB5+perf+top+cpu44+20220513-1.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27977/Bole+IGW+NPB5+perf+top+cpu44+20220513-1.png)



Attachment: Bole+IGW+NPB5+perf+top+cpu48+20220526.png

![Bole+IGW+NPB5+perf+top+cpu48+20220526.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/28269/Bole+IGW+NPB5+perf+top+cpu48+20220526.png)



Attachment: Bole+IGW+NPB5-QUIC.SNI_session.csv

[Bole+IGW+NPB5-QUIC.SNI_session.csv](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27669/Bole+IGW+NPB5-QUIC.SNI_session.csv)



Attachment: Bole+IGW+NPB5-SSL.SNI_session.csv

[Bole+IGW+NPB5-SSL.SNI_session.csv](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27670/Bole+IGW+NPB5-SSL.SNI_session.csv)



Attachment: Bole+IGW+NPB5+sysinfo.log

[Bole+IGW+NPB5+sysinfo.log](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27696/Bole+IGW+NPB5+sysinfo.log)



Attachment: Bole+IGW+NPB5+throughtput+20220513.png

![Bole+IGW+NPB5+throughtput+20220513.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27972/Bole+IGW+NPB5+throughtput+20220513.png)



Attachment: E21-sapp-overload-protection-bypass-20220427~20220503.png

![E21-sapp-overload-protection-bypass-20220427~20220503.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27750/E21-sapp-overload-protection-bypass-20220427~20220503.png)



Attachment: e21-version-21.11-dup-fqdn-object.txt

[e21-version-21.11-dup-fqdn-object.txt](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27643/e21-version-21.11-dup-fqdn-object.txt)



Attachment: image-2022-04-27-17-12-00-094.png

![image-2022-04-27-17-12-00-094.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27599/image-2022-04-27-17-12-00-094.png)



Attachment: image-2022-04-27-17-14-25-901.png

![image-2022-04-27-17-14-25-901.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27600/image-2022-04-27-17-14-25-901.png)



Attachment: image-2022-04-27-17-15-02-774.png

![image-2022-04-27-17-15-02-774.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27601/image-2022-04-27-17-15-02-774.png)



Attachment: image-2022-04-27-17-17-32-639.png

![image-2022-04-27-17-17-32-639.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27602/image-2022-04-27-17-17-32-639.png)



Attachment: image-2022-04-27-17-18-43-458.png

![image-2022-04-27-17-18-43-458.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27604/image-2022-04-27-17-18-43-458.png)



Attachment: image-2022-04-27-17-19-09-881.png

![image-2022-04-27-17-19-09-881.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27605/image-2022-04-27-17-19-09-881.png)



Attachment: image-2022-04-27-17-35-20-050.png

![image-2022-04-27-17-35-20-050.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27615/image-2022-04-27-17-35-20-050.png)



Attachment: image-2022-04-27-17-36-03-049.png

![image-2022-04-27-17-36-03-049.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27616/image-2022-04-27-17-36-03-049.png)



Attachment: image-2022-04-27-17-53-42-859.png

![image-2022-04-27-17-53-42-859.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27617/image-2022-04-27-17-53-42-859.png)



Attachment: image-2022-04-28-15-43-09-227.png

![image-2022-04-28-15-43-09-227.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27656/image-2022-04-28-15-43-09-227.png)



Attachment: image-2022-04-29-09-44-38-025.png

![image-2022-04-29-09-44-38-025.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27682/image-2022-04-29-09-44-38-025.png)



Attachment: image-2022-05-06-18-15-08-649.png

![image-2022-05-06-18-15-08-649.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27779/image-2022-05-06-18-15-08-649.png)



Attachment: image-2022-05-07-16-12-05-474.png

![image-2022-05-07-16-12-05-474.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27789/image-2022-05-07-16-12-05-474.png)



Attachment: image-2022-05-16-15-32-36-510.png

![image-2022-05-16-15-32-36-510.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/28003/image-2022-05-16-15-32-36-510.png)



Attachment: image-2022-05-16-15-33-27-227.png

![image-2022-05-16-15-33-27-227.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/28006/image-2022-05-16-15-33-27-227.png)



Attachment: image-2022-05-24-17-06-33-618.png

![image-2022-05-24-17-06-33-618.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/28229/image-2022-05-24-17-06-33-618.png)



Attachment: image-2022-05-31-09-44-22-020.png

![image-2022-05-31-09-44-22-020.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/28335/image-2022-05-31-09-44-22-020.png)



Attachment: image-2022-05-31-09-45-23-952.png

![image-2022-05-31-09-45-23-952.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/28336/image-2022-05-31-09-45-23-952.png)



Attachment: image-2022-05-31-09-46-02-588.png

![image-2022-05-31-09-46-02-588.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/28337/image-2022-05-31-09-46-02-588.png)



Attachment: image-2022-06-10-15-36-23-768.png

![image-2022-06-10-15-36-23-768.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/28628/image-2022-06-10-15-36-23-768.png)



Attachment: image-2022-06-10-15-37-33-666.png

![image-2022-06-10-15-37-33-666.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/28629/image-2022-06-10-15-37-33-666.png)



Attachment: image-2022-06-10-15-38-10-175.png

![image-2022-06-10-15-38-10-175.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/28630/image-2022-06-10-15-38-10-175.png)



Attachment: L7protocal_sessions-all-NPB.csv

[L7protocal_sessions-all-NPB.csv](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27660/L7protocal_sessions-all-NPB.csv)



Attachment: L7protocal_sessions-all-NPB-new.csv

[L7protocal_sessions-all-NPB-new.csv](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27661/L7protocal_sessions-all-NPB-new.csv)



Attachment: L7protocal_sessions-Bole+IGW+NPB2.csv

[L7protocal_sessions-Bole+IGW+NPB2.csv](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27662/L7protocal_sessions-Bole+IGW+NPB2.csv)



Attachment: L7protocal_sessions-Bole+IGW+NPB5.csv

[L7protocal_sessions-Bole+IGW+NPB5.csv](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27663/L7protocal_sessions-Bole+IGW+NPB5.csv)



Attachment: L7protocol_sessions-Bole+IGW+NPB5.pdf

[L7protocol_sessions-Bole+IGW+NPB5.pdf](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27664/L7protocol_sessions-Bole+IGW+NPB5.pdf)



Attachment: L7protocol_ssesion-all-NPB.pdf

[L7protocol_ssesion-all-NPB.pdf](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27665/L7protocol_ssesion-all-NPB.pdf)



Attachment: L7protocol_ssesion-Bole+IGW+NPB2.pdf

[L7protocol_ssesion-Bole+IGW+NPB2.pdf](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27666/L7protocol_ssesion-Bole+IGW+NPB2.pdf)



Attachment: perf.svg

[perf.svg](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27606/perf.svg)



Attachment: perf33.svg

[perf33.svg](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27607/perf33.svg)



Attachment: perf33-1.svg

[perf33-1.svg](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27610/perf33-1.svg)



Attachment: perf37.svg

[perf37.svg](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27608/perf37.svg)



Attachment: perf40.svg

[perf40.svg](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27609/perf40.svg)



Attachment: securityEvents-target.xlsx

[securityEvents-target.xlsx](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27544/securityEvents-target.xlsx)



Attachment: sessionRecords-deny-target.com-failed.xlsx

[sessionRecords-deny-target.com-failed.xlsx](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27613/sessionRecords-deny-target.com-failed.xlsx)



Attachment: sessionRecords+-target-clientip.xlsx

[sessionRecords+-target-clientip.xlsx](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27543/sessionRecords+-target-clientip.xlsx)



Attachment: target.com-196.188.136.150-151.101.2.187.443.pcap

[target.com-196.188.136.150-151.101.2.187.443.pcap](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27614/target.com-196.188.136.150-151.101.2.187.443.pcap)



Attachment: tsg_static_maat.txt

[tsg_static_maat.txt](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27642/tsg_static_maat.txt)



Attachment: XJ-192.175-app_sketch_maat.txt

[XJ-192.175-app_sketch_maat.txt](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27720/XJ-192.175-app_sketch_maat.txt)



Attachment: XJ-192.175-app-maat.png

![XJ-192.175-app-maat.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27716/XJ-192.175-app-maat.png)



Attachment: XJ-192.175-monit_device.png

![XJ-192.175-monit_device.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27712/XJ-192.175-monit_device.png)



Attachment: XJ-192.175-perf-cpu-6.svg

[XJ-192.175-perf-cpu-6.svg](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27723/XJ-192.175-perf-cpu-6.svg)



Attachment: XJ-192.175-perf-top-cpu6.png

![XJ-192.175-perf-top-cpu6.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27717/XJ-192.175-perf-top-cpu6.png)



Attachment: XJ-192.175-sysinfo.png

![XJ-192.175-sysinfo.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27714/XJ-192.175-sysinfo.png)



Attachment: XJ-192.175-sysinfo.txt

[XJ-192.175-sysinfo.txt](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27718/XJ-192.175-sysinfo.txt)



Attachment: XJ-192.175-top-all-cpu.png

![XJ-192.175-top-all-cpu.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27713/XJ-192.175-top-all-cpu.png)



Attachment: XJ-192.175-tsg_static_maat.txt

[XJ-192.175-tsg_static_maat.txt](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27719/XJ-192.175-tsg_static_maat.txt)



Attachment: XJ-192.175-tsg-maat.png

![XJ-192.175-tsg-maat.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27715/XJ-192.175-tsg-maat.png)



Attachment: XXG-40.82-load-app-maat-all-cpu.png

![XXG-40.82-load-app-maat-all-cpu.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27742/XXG-40.82-load-app-maat-all-cpu.png)



Attachment: XXG-40.82-load-app-maat-new-link.png

![XXG-40.82-load-app-maat-new-link.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27745/XXG-40.82-load-app-maat-new-link.png)



Attachment: XXG-40.82-load-app-maat-perf-top-cpu14.png

![XXG-40.82-load-app-maat-perf-top-cpu14.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27743/XXG-40.82-load-app-maat-perf-top-cpu14.png)



Attachment: XXG-40.82-load-app-maat-sysinfo.txt

[XXG-40.82-load-app-maat-sysinfo.txt](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27744/XXG-40.82-load-app-maat-sysinfo.txt)



Attachment: XXG-40.82-no-app-maat--all-cpu.png

![XXG-40.82-no-app-maat--all-cpu.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27746/XXG-40.82-no-app-maat--all-cpu.png)



Attachment: XXG-40.82-no-app-maat-new-link.png

![XXG-40.82-no-app-maat-new-link.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27748/XXG-40.82-no-app-maat-new-link.png)



Attachment: XXG-40.82-no-app-maat-perf-top-cpu14.png

![XXG-40.82-no-app-maat-perf-top-cpu14.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27747/XXG-40.82-no-app-maat-perf-top-cpu14.png)



Attachment: XXG-40.82-no-app-maat-sysinfo.txt

[XXG-40.82-no-app-maat-sysinfo.txt](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27749/XXG-40.82-no-app-maat-sysinfo.txt)



Attachment: XXG-9140.sapp.bypass.png

![XXG-9140.sapp.bypass.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/28933/XXG-9140.sapp.bypass.png)



Attachment: 微信图片_20220426170327.png

![微信图片_20220426170327.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27547/微信图片_20220426170327.png)



Attachment: 微信图片_20220426170336.png

![微信图片_20220426170336.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27546/微信图片_20220426170336.png)



Attachment: 微信图片_20220426170342.png

![微信图片_20220426170342.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27545/微信图片_20220426170342.png)



Attachment: 微信图片_20220426170353.png

![微信图片_20220426170353.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27549/微信图片_20220426170353.png)



Attachment: 微信图片_20220426170400-1.png

![微信图片_20220426170400-1.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27548/微信图片_20220426170400-1.png)



Attachment: 微信图片_20220426170415.png

![微信图片_20220426170415.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27551/微信图片_20220426170415.png)



Attachment: 微信图片_20220426170429.png

![微信图片_20220426170429.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27550/微信图片_20220426170429.png)



Attachment: 微信图片_20220426170433.png

![微信图片_20220426170433.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27555/微信图片_20220426170433.png)



Attachment: 微信图片_20220426170438.png

![微信图片_20220426170438.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27554/微信图片_20220426170438.png)



Attachment: 微信图片_20220426170442.png

![微信图片_20220426170442.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27553/微信图片_20220426170442.png)



Attachment: 微信图片_20220426170446.png

![微信图片_20220426170446.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/27552/微信图片_20220426170446.png)