158 lines
6.0 KiB
Markdown
158 lines
6.0 KiB
Markdown
# 自检拦截策略下载4M、16M、64M文件的用例不通过
|
||
|
||
| ID | Creation Date | Assignee | Status |
|
||
|----|----------------|----------|--------|
|
||
| OMPUB-1028 | 2023-10-12T11:44:49.000+0800 | 付明卫 | 已关闭 |
|
||
|
||
|
||
---
|
||
|
||
设备： nfs-tsgx01（10.3.60.7）
|
||
|
||
OS版本：v23.07.18-591aed7 (TSGXNXR620G40R01P0906)
|
||
|
||
自检执行结果如下：
|
||
{code:java}
|
||
test_firewallBypass_ssl ... ok
|
||
test_firewallIntercept_ssl ... ok
|
||
test_firewallIntercept_sslCerterrExpired ... ok
|
||
test_firewallIntercept_sslCerterrSelfsigned ... ok
|
||
test_firewallIntercept_sslCerterrUntrustedroot ... ok
|
||
test_proxyRedirect_ssl ... ok
|
||
test_proxyBlock_ssl ... ok
|
||
test_proxyReplace_ssl ... ok
|
||
test_proxyHijack_ssl ... ok
|
||
test_proxyInsert_ssl ... ok
|
||
test_proxyRedirect_http ... ok
|
||
test_proxyBlock_http ... ok
|
||
test_proxyReplace_http ... ok
|
||
test_proxyHijack_http ... ok
|
||
test_proxyInsert_http ... ok
|
||
test_firewallAllow_http ... ok
|
||
test_firewallDenyDrop_http ... ok
|
||
test_firewallDenyReset_http ... ok
|
||
test_firewallDenyBlock_http ... ok
|
||
test_firewallAllow_ssl ... ok
|
||
test_firewallDenyDrop_ssl ... ok
|
||
test_firewallDenyReset_ssl ... ok
|
||
test_firewallDenyDrop_dns ... ok
|
||
test_firewallDenyRedirectA_dns ... ok
|
||
test_firewallDenyRedirectAAAA_dns ... ok
|
||
test_firewallDenyRedirectARangeTTL_dns ... ok
|
||
test_firewallDenyRedirectAAAARangeTTL_dns ... ok
|
||
test_firewallIntercept_sslDownloadSize1k ... ok
|
||
test_firewallIntercept_sslDownloadSize4k ... ok
|
||
test_firewallIntercept_sslDownloadSize16k ... ok
|
||
test_firewallIntercept_sslDownloadSize64k ... ok
|
||
test_firewallIntercept_sslDownloadSize256k ... ok
|
||
test_firewallIntercept_sslDownloadSize1M ... ok
|
||
test_firewallIntercept_sslDownloadSize4M ... FAIL
|
||
test_firewallIntercept_sslDownloadSize16M ... FAIL
|
||
test_firewallIntercept_sslDownloadSize64M ... FAIL
|
||
test_firewallDenyResetFilterHost_http ... ok
|
||
test_firewallDenyResetFilterURL_http ... ok
|
||
test_proxyDenyFilterHost_http ... ok
|
||
test_proxyDenyFilterURL_http ... ok {code}**fu_mingwei** commented on *2023-10-12T12:01:38.372+0800*:
|
||
|
||
通过单独执行test_firewallINtercept_sslDownloadSize4M case来复现上述现象，命令如下：
|
||
* kubectl exec -n tsg-os-system -it dign-client-clxz8 -c dign-client /bin/sh
|
||
* curl -v [https://testing-download.badssl.selftest.gdnt-cloud.website/resources/4M] --resolve "testing-download.badssl.selftest.gdnt-cloud.website:443:192.0.2.110" -k --output temp
|
||
|
||
通过iptables -x -v -L KUBE-FORWARD --line-numbers查看发现数据包命中丢包策略。
|
||
{code:java}
|
||
[root@nfs-tsgx01 ~]# iptables -x -v -L KUBE-FORWARD --line-numbers
|
||
Chain KUBE-FORWARD (1 references)
|
||
num pkts bytes target prot opt in out source destination
|
||
1 123 71318 DROP all -- any any anywhere anywhere ctstate INVALID
|
||
2 0 0 ACCEPT all -- any any anywhere anywhere /* kubernetes forwarding rules */ mark match 0x4000/0x4000
|
||
3 22517 102350860 ACCEPT all -- any any anywhere anywhere /* kubernetes forwarding conntrack rule */ ctstate RELATED,ESTABLISHED
|
||
{code}
|
||
通过命令conntrack -L | grep 192.0.2发现自检的tcp连接处于time_wait状态。
|
||
{code:java}
|
||
[root@nfs-tsgx01 ~]# conntrack -L | grep 192.0.2
|
||
tcp 6 115 TIME_WAIT src=192.0.2.218 dst=192.0.2.110 sport=52558 dport=443 src=192.0.2.110 dst=192.0.2.218 sport=443 dport=52558 [ASSURED] mark=0 use=1
|
||
conntrack v1.4.4 (conntrack-tools): 1041 flow entries have been shown. {code}
|
||
|
||
|
||
|
||
|
||
---
|
||
|
||
**fu_mingwei** commented on *2023-10-12T12:49:37.870+0800*:
|
||
|
||
解决方法1:
|
||
|
||
执行命令：
|
||
* iptables -I FORWARD -d 192.0.2.0/24 -j ACCEPT
|
||
|
||
在FORWARD链中插入接收目的网络为192.0.2.0/24包的规则。
|
||
|
||
解决方法2：
|
||
|
||
执行命令：
|
||
* iptables -t raw -I PREROUTING -i br_dign_c -j NOTRACK
|
||
* iptables -t raw -I PREROUTING -i br_dign_s -j NOTRACK
|
||
|
||
|
||
|
||
---
|
||
|
||
**fu_mingwei** commented on *2023-10-12T18:46:46.076+0800*:
|
||
|
||
上述的解决方法2已经在NetworkManager dispatcher中添加相应脚本执行。
|
||
|
||
通过排查Networkmanager dispatcher脚本是否执行的过程中发现在描述中机器上NetworkManager-dispatcher service为disable状态。
|
||
|
||
!image-2023-10-12-18-46-37-480.png!
|
||
|
||
通知执行命令mount /dev/sda4 /tmp/rootfs发现NetworkManger-dispatcher的状态是enable
|
||
|
||
!image-2023-10-12-19-44-57-743.png!
|
||
|
||
通知查看os的overlay目录发现dbus-org.freedesktop.nm-dispatcher.service存在
|
||
|
||
!image-2023-10-12-19-48-25-708.png!
|
||
|
||
|
||
|
||
---
|
||
|
||
**fu_mingwei** commented on *2023-10-12T20:14:50.909+0800*:
|
||
|
||
结论：
|
||
|
||
上述问题是由于未添加以下iptables规则而导致的：{{{}iptables -t raw -I PREROUTING -i br_dign_c -j NOTRACK{}}}和{{{}iptables -t raw -I PREROUTING -i br_dign_c -j NOTRACK{}}}。
|
||
|
||
添加上述iptables规则的目的是为了防止conntrack将代理修复出来的两个连接识别为同一个conntrack信息。修复出来的两个连接在关闭时并不同时，这导致了conntrack将没有同时关闭的两条连接误认为是TIME_WAIT状态，并被K8S添加的iptables规则丢弃。
|
||
|
||
上述iptables规则是由NetworkManger-dispatcher服务添加的。然而，由于存在NetworkManger-dispatcher服务的残留文件等原因，导致NetworkManger-dispatcher服务处于禁用状态。当NetworkManger-dispatcher服务处于禁用状态时，导致上述iptables规则无法成功添加。
|
||
|
||
|
||
|
||
|
||
|
||
---
|
||
|
||
|
||
|
||
# Attachments
|
||
|
||
Attachment: image-2023-10-12-18-46-37-480.png
|
||
|
||
|
||
|
||
|
||
|
||
Attachment: image-2023-10-12-19-44-57-743.png
|
||
|
||
|
||
|
||
|
||
|
||
Attachment: image-2023-10-12-19-48-25-708.png
|
||
|
||
|
||
|
||
|
||
|