admin/geedge-jira
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
master
geedge-jira/md/OMPUB-1226.md

226 lines
8.0 KiB
Markdown
Raw Permalink Normal View History

first
2025-09-14 21:52:36 +00:00
# 【WMS-UTR】smtp/pop3/imap监测日志中，decoded as为mail的日志仅占一半，且from，to地址解析比例低
| ID | Creation Date | Assignee | Status |
|----|----------------|----------|--------|
| OMPUB-1226 | 2024-04-14T16:09:33.000+0800 | 刘洋 | 已关闭 |
---
[~liuyang] 洋姐，监测邮件的策略中配置了smtp、pop3、imap三种协议
!image-2024-04-14-14-35-49-524.png|width=530,height=338!
统计了20240414 10:53:06~10:54:01共100000条mail的监测日志：
* decoded as成mail的有58298条，其余解码为base/http/ssl。
* mail协议中出现mail from/to等邮箱地址的数量统计如下
!image-2024-04-14-14-35-22-882.png!
日志见附件的压缩文件**yangwei** commented on *2024-04-15T09:59:51.732+0800*:
导出的100000条监测日志中，decode_as分布如下：
* MAIL 58298
* BASE 41687
** app识别结果均为pop,imap,smtp
** server 端口分布如下：
***
||server port||sessions||app||
|25|36020|smtp:36020
imap:1|
|110|2956|pop3|
|143|2709|smtp:4
imap:2705|
|587|1|smtp|
|80|1|smtp|
*
** 上述端口均为mail常用服务端端口，初步推测decode_as为BASE的原因同https://jira.geedge.net/browse/TSG-19842
* HTTP 3
** app识别结果均为http.imap或者http.pop3
** UA均为 python-socks/2.3.0，推测为http代理
* SSL 12
** app识别结果均为pop3.ssl或者imap.ssl，应该均为STARTLS
 
---
**yangwei** commented on *2024-04-15T17:42:13.527+0800*:
Decode_as为BASE的分析
* 对于app识别为mail(in smtp, pop, imap)，decode_as为BASE的会话，通过在京版网关下发server port in (25,110,143,587) and ip_protocol==tcp的监测策略，并开启packet capture，3小时记录106个会话，Decode AS分布如下：
** BASE 37 占比34.9%
** MAIL 69 占比65.69%
*** C2S单向流 35041，占比53%
*** S2C单向流 24，占比0.03%
* 进一步分析BASE的会话，app识别结果如下：
** !image-2024-04-15-17-42-18-348.png|width=745,height=164!
** app识别为VPN的会话结果无误，原因为VPN服务使用常见邮件端口。
** app识别为imap的会话，典型的传输内容如下：
** !image-2024-04-15-17-42-26-655.png|width=1748,height=250!
** 上图会话传输的内容为qqmail服务器的欢迎（greeting）消息，客户端与服务端握手成功后，服务端主动返回其支持的功能（Capabilities），包括不同的认证方法和命令扩展。
** 上述会话周期性出现，被第三方DPI识别为imap并在app字段标注，但由于未传输mail日志相关字段，decode_as将被标注为BASE。
* 分析decode_as的会话，传输的内容如下：
** !image-2024-04-15-17-48-17-008.png|width=609,height=273!
** 上述会话decode_as会被标注为MAIL，但是mail_xx字段全部为空，原因是mail decoder在出现STARTTLS后，会主动通知firewall插件该状态，但并未出现已定义的相关日志字段。
{*}修改建议{*}：参照SSL，新增mail.starttls flags日志字段，用于标注MAIL会话中STARTTLS的行为
---
**niuxiang** commented on *2024-04-16T02:01:16.917+0800*:
UTR现场MAIL数据包下载地址
https://de.files.gdnt-cloud.com/d/712121426ea244bcaf40/
---
**yangwei** commented on *2024-04-16T11:25:56.027+0800*:
现场返回两个pcapng数据包，功能端处理后结果如下：
*数据包 10164_port25.pcapng 259.7MB*
* TCP会话：12670
** c2s 11347 89.5%
** s2c 971 7.5%
** bidirectional 2.7%
* 有效传输数据会话（包数>3,字节数>5）:5775
** *c2s* 4936 *85.5%*
** *s2c* 549 *9.5%*
** *bidirectional* 290 *5%*
* 解析为SMTP 5017, 占有效传输数据会话 86.8%
** *STARTTLS* 1298 *占SMTP会话 25.8%*
** 含MAIL_FROM_CMD 3410，占SMTP会话67.9%
** 含MAIL_FROM 248, 占SMTP会话 4.9%
** 含MAIL_TO_CMD 3328，占SMTP会话 66.3%
** 含MAIL_TO 246, 占SMTP会话 4.9%
*** 含MAIL_FROM_CMD但是不含MAIL_FROM传输内容如下：
*** SMTP命令行交互后，被服务端RST
*** !image-2024-04-16-12-16-06-915.png|width=859,height=218!
** *无上述MAIL字段 309，占SMTP会话 6%*
*** 无mail收发件人字段传输的内容如下：
**** SMTP 单向流 ，客户端发起EHLO
**** !image-2024-04-16-11-35-08-052.png|width=988,height=183!
**** SMTP双向流
**** !image-2024-04-16-11-36-32-762.png|width=573,height=189!
 
*数据包 10164_port25port110port143.pcapng 303.7MB*
* TCP会话：17963
** c2s 16363 91%
** s2c 1221 6.9%
** bidirectional 379 2.1%
* 有效传输数据会话（包数>3,字节数>5）:8343
** *c2s* 7315 *87.6%*
** *s2c* 684 *8.2%*
** *bidirectional* 344 *4.2%*
* 解析为MAIL 6714, 占有效传输数据会话 80.4%
** *STARTTLS* 2124 *占MAIL会话 31.6%*
** 含MAIL_FROM_CMD 3758，占MAIL会话55.9%
** 含MAIL_FROM 123, 占MAIL会话 1.8%
** 含MAIL_TO_CMD 3681，占MAIL会话 54.8%
** 含MAIL_TO 122, 占MAIL会话 1.8%
** {*}不含上述MAIL字段{*}会话832，占MAIL会话 *12.3%*
*** 无mail收发件人字段传输的内容如下：
**** pop3 单向流，仅传输用户名密码
*** !image-2024-04-16-12-00-35-035.png|width=1375,height=322!
*** IMAP单向流，客户端登入登出
*** !image-2024-04-16-12-03-25-954.png|width=834,height=186!
 
*初步结论*
* 监测mail，日志中decode as为base的原因
** 对于MAIL.STARTTLS会话，当前监测日志中Decode As会被填充为BASE，结合Application识别结果可以判断
** 现场采样的数据包，STARTTLS会话比例在25%~30%
* decode as为MAIL，但是收发件人为空的原因
** Decode As为MAIL的会话，存在正常交互，但是不传输MAIL收发件人信息的行为（按捕包+采样统计约占MAIL会话的10%~12%）
** 日志中仅包含MAIL_FROM_CMD不包含MAIL_FROM的情况，通常为命令行交互过程中，未进入到传输MAIL内容的流程即被服务端中止
** 单向流情况下，单侧不包含MAIL收发件人信息
*** SMTP S2C侧
*** POP和IMAP C2S侧
---
attachment link
2025-09-14 22:26:17 +00:00
# Attachments
first
2025-09-14 21:52:36 +00:00
attachment link
2025-09-14 22:26:17 +00:00
Attachment: image-2024-04-14-14-35-22-882.png
attachment link
2025-09-14 22:27:11 +00:00
attachment link
2025-09-14 22:26:17 +00:00
![image-2024-04-14-14-35-22-882.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/55173/image-2024-04-14-14-35-22-882.png)
first
2025-09-14 21:52:36 +00:00
attachment link
2025-09-14 22:26:17 +00:00
Attachment: image-2024-04-14-14-35-49-524.png
attachment link
2025-09-14 22:27:11 +00:00
attachment link
2025-09-14 22:26:17 +00:00
![image-2024-04-14-14-35-49-524.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/55172/image-2024-04-14-14-35-49-524.png)
first
2025-09-14 21:52:36 +00:00
attachment link
2025-09-14 22:26:17 +00:00
Attachment: image-2024-04-15-17-42-18-348.png
attachment link
2025-09-14 22:27:11 +00:00
attachment link
2025-09-14 22:26:17 +00:00
![image-2024-04-15-17-42-18-348.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/55280/image-2024-04-15-17-42-18-348.png)
first
2025-09-14 21:52:36 +00:00
attachment link
2025-09-14 22:26:17 +00:00
Attachment: image-2024-04-15-17-42-26-655.png
attachment link
2025-09-14 22:27:11 +00:00
attachment link
2025-09-14 22:26:17 +00:00
![image-2024-04-15-17-42-26-655.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/55279/image-2024-04-15-17-42-26-655.png)
first
2025-09-14 21:52:36 +00:00
attachment link
2025-09-14 22:26:17 +00:00
Attachment: image-2024-04-15-17-48-17-008.png
attachment link
2025-09-14 22:27:11 +00:00
attachment link
2025-09-14 22:26:17 +00:00
![image-2024-04-15-17-48-17-008.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/55281/image-2024-04-15-17-48-17-008.png)
first
2025-09-14 21:52:36 +00:00
attachment link
2025-09-14 22:26:17 +00:00
Attachment: image-2024-04-16-11-35-08-052.png
attachment link
2025-09-14 22:27:11 +00:00
attachment link
2025-09-14 22:26:17 +00:00
![image-2024-04-16-11-35-08-052.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/55323/image-2024-04-16-11-35-08-052.png)
first
2025-09-14 21:52:36 +00:00
attachment link
2025-09-14 22:26:17 +00:00
Attachment: image-2024-04-16-11-36-32-762.png
attachment link
2025-09-14 22:27:11 +00:00
attachment link
2025-09-14 22:26:17 +00:00
![image-2024-04-16-11-36-32-762.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/55324/image-2024-04-16-11-36-32-762.png)
first
2025-09-14 21:52:36 +00:00
attachment link
2025-09-14 22:26:17 +00:00
Attachment: image-2024-04-16-12-00-35-035.png
attachment link
2025-09-14 22:27:11 +00:00
attachment link
2025-09-14 22:26:17 +00:00
![image-2024-04-16-12-00-35-035.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/55326/image-2024-04-16-12-00-35-035.png)
Attachment: image-2024-04-16-12-03-25-954.png
attachment link
2025-09-14 22:27:11 +00:00
attachment link
2025-09-14 22:26:17 +00:00
![image-2024-04-16-12-03-25-954.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/55325/image-2024-04-16-12-03-25-954.png)
Attachment: image-2024-04-16-12-16-06-915.png
attachment link
2025-09-14 22:27:11 +00:00
attachment link
2025-09-14 22:26:17 +00:00
![image-2024-04-16-12-16-06-915.png](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/55327/image-2024-04-16-12-16-06-915.png)
Attachment: monitor_event20240414055548.rar
attachment link
2025-09-14 22:27:11 +00:00
attachment link
2025-09-14 22:26:17 +00:00
[monitor_event20240414055548.rar](https://gfwleak.exec.li/admin/geedge-jira/raw/branch/master/attachment/55175/monitor_event20240414055548.rar)
first
2025-09-14 21:52:36 +00:00
Reference in New Issue Copy Permalink