6.0 KiB
自检拦截策略下载4M、16M、64M文件的用例不通过
| ID | Creation Date | Assignee | Status |
|---|---|---|---|
| OMPUB-1028 | 2023-10-12T11:44:49.000+0800 | 付明卫 | 已关闭 |
设备： nfs-tsgx01（10.3.60.7）
OS版本：v23.07.18-591aed7 (TSGXNXR620G40R01P0906)
自检执行结果如下： {code:java} test_firewallBypass_ssl ... ok test_firewallIntercept_ssl ... ok test_firewallIntercept_sslCerterrExpired ... ok test_firewallIntercept_sslCerterrSelfsigned ... ok test_firewallIntercept_sslCerterrUntrustedroot ... ok test_proxyRedirect_ssl ... ok test_proxyBlock_ssl ... ok test_proxyReplace_ssl ... ok test_proxyHijack_ssl ... ok test_proxyInsert_ssl ... ok test_proxyRedirect_http ... ok test_proxyBlock_http ... ok test_proxyReplace_http ... ok test_proxyHijack_http ... ok test_proxyInsert_http ... ok test_firewallAllow_http ... ok test_firewallDenyDrop_http ... ok test_firewallDenyReset_http ... ok test_firewallDenyBlock_http ... ok test_firewallAllow_ssl ... ok test_firewallDenyDrop_ssl ... ok test_firewallDenyReset_ssl ... ok test_firewallDenyDrop_dns ... ok test_firewallDenyRedirectA_dns ... ok test_firewallDenyRedirectAAAA_dns ... ok test_firewallDenyRedirectARangeTTL_dns ... ok test_firewallDenyRedirectAAAARangeTTL_dns ... ok test_firewallIntercept_sslDownloadSize1k ... ok test_firewallIntercept_sslDownloadSize4k ... ok test_firewallIntercept_sslDownloadSize16k ... ok test_firewallIntercept_sslDownloadSize64k ... ok test_firewallIntercept_sslDownloadSize256k ... ok test_firewallIntercept_sslDownloadSize1M ... ok test_firewallIntercept_sslDownloadSize4M ... FAIL test_firewallIntercept_sslDownloadSize16M ... FAIL test_firewallIntercept_sslDownloadSize64M ... FAIL test_firewallDenyResetFilterHost_http ... ok test_firewallDenyResetFilterURL_http ... ok test_proxyDenyFilterHost_http ... ok test_proxyDenyFilterURL_http ... ok {code}fu_mingwei commented on 2023-10-12T12:01:38.372+0800:
通过单独执行test_firewallINtercept_sslDownloadSize4M case来复现上述现象，命令如下：
- kubectl exec -n tsg-os-system -it dign-client-clxz8 -c dign-client /bin/sh
- curl -v [https://testing-download.badssl.selftest.gdnt-cloud.website/resources/4M] --resolve "testing-download.badssl.selftest.gdnt-cloud.website:443:192.0.2.110" -k --output temp
通过iptables -x -v -L KUBE-FORWARD --line-numbers查看发现数据包命中丢包策略。 {code:java} [root@nfs-tsgx01 ~]# iptables -x -v -L KUBE-FORWARD --line-numbers Chain KUBE-FORWARD (1 references) num pkts bytes target prot opt in out source destination 1 123 71318 DROP all -- any any anywhere anywhere ctstate INVALID 2 0 0 ACCEPT all -- any any anywhere anywhere /* kubernetes forwarding rules / mark match 0x4000/0x4000 3 22517 102350860 ACCEPT all -- any any anywhere anywhere / kubernetes forwarding conntrack rule */ ctstate RELATED,ESTABLISHED {code} 通过命令conntrack -L | grep 192.0.2发现自检的tcp连接处于time_wait状态。 {code:java} [root@nfs-tsgx01 ~]# conntrack -L | grep 192.0.2 tcp 6 115 TIME_WAIT src=192.0.2.218 dst=192.0.2.110 sport=52558 dport=443 src=192.0.2.110 dst=192.0.2.218 sport=443 dport=52558 [ASSURED] mark=0 use=1 conntrack v1.4.4 (conntrack-tools): 1041 flow entries have been shown. {code}
fu_mingwei commented on 2023-10-12T12:49:37.870+0800:
解决方法1:
执行命令：
- iptables -I FORWARD -d 192.0.2.0/24 -j ACCEPT
在FORWARD链中插入接收目的网络为192.0.2.0/24包的规则。
解决方法2：
执行命令：
- iptables -t raw -I PREROUTING -i br_dign_c -j NOTRACK
- iptables -t raw -I PREROUTING -i br_dign_s -j NOTRACK
fu_mingwei commented on 2023-10-12T18:46:46.076+0800:
上述的解决方法2已经在NetworkManager dispatcher中添加相应脚本执行。
通过排查Networkmanager dispatcher脚本是否执行的过程中发现在描述中机器上NetworkManager-dispatcher service为disable状态。
!image-2023-10-12-18-46-37-480.png!
通知执行命令mount /dev/sda4 /tmp/rootfs发现NetworkManger-dispatcher的状态是enable
!image-2023-10-12-19-44-57-743.png!
通知查看os的overlay目录发现dbus-org.freedesktop.nm-dispatcher.service存在
!image-2023-10-12-19-48-25-708.png!
fu_mingwei commented on 2023-10-12T20:14:50.909+0800:
结论：
上述问题是由于未添加以下iptables规则而导致的：{{{}iptables -t raw -I PREROUTING -i br_dign_c -j NOTRACK{}}}和{{{}iptables -t raw -I PREROUTING -i br_dign_c -j NOTRACK{}}}。
添加上述iptables规则的目的是为了防止conntrack将代理修复出来的两个连接识别为同一个conntrack信息。修复出来的两个连接在关闭时并不同时，这导致了conntrack将没有同时关闭的两条连接误认为是TIME_WAIT状态，并被K8S添加的iptables规则丢弃。
上述iptables规则是由NetworkManger-dispatcher服务添加的。然而，由于存在NetworkManger-dispatcher服务的残留文件等原因，导致NetworkManger-dispatcher服务处于禁用状态。当NetworkManger-dispatcher服务处于禁用状态时，导致上述iptables规则无法成功添加。
Attachments
Attachment: image-2023-10-12-18-46-37-480.png
Attachment: image-2023-10-12-19-44-57-743.png
Attachment: image-2023-10-12-19-48-25-708.png
