123 lines
3.8 KiB
Markdown
123 lines
3.8 KiB
Markdown
# 【E21现场】针对Dos Events数据，能否快速统计出针对某一个destination ip 的top N source ip
|
||
|
||
| ID | Creation Date | Assignee | Status |
|
||
|----|----------------|----------|--------|
|
||
| OMPUB-873 | 2023-03-28T02:35:39.000+0800 | | 完成 |
|
||
|
||
|
||
---
|
||
|
||
针对Dos Events数据，能否快速统计出指定时间段内，某一个destination ip 的top N source ip 信息嘛。
|
||
|
||
目前在E现场统计2022-09-01~2023-03-16时间段内，Dos Events里Severity=Critical的日志信息，
|
||
|
||
Top 2 Destination IP分别为：
|
||
|
||
197.156.74.223：236次
|
||
|
||
197.156.74.222：166次
|
||
|
||
|
||
|
||
搜索指定destination IP的日志，得到的是Source IPs列表，里面甚至有很多是Private IP,如何能快速统计出指定时间段具体某个destination ip 的top N source ip 及出现的总次数。**doufenghu** commented on *2023-03-30T14:58:25.140+0800*:
|
||
|
||
* 通过SQL查询获取结果
|
||
|
||
{code:java}
|
||
select item, sum(count) as count from ( select arrayJoin(items) as item,count from ( select splitByString(',', source_ip_list) as items, count(*) as count from dos_event lt where destination_ip = '27.50.64.2' and notEmpty(source_ip_list) and start_time >= toUnixTimestamp(parseDateTimeBestEffort(toString('2023-03-10 00:00:00+03:00'))) AND start_time < toUnixTimestamp(parseDateTimeBestEffort(toString('2023-03-26 00:00:00+03:00'))) group by source_ip_list ) ) group by item order by count desc limit 10
|
||
{code}
|
||
* 后续版本支持：
|
||
** 方式1: 可在23.04 Source IPs 定义业务逻辑类型为Array，作为多值列统计（基于v23.03业务逻辑类型定义支持）
|
||
** 方式2: Report Log Type增加 DoS Events ，增加预置数据集
|
||
|
||
|
||
|
||
---
|
||
|
||
**liuju** commented on *2023-03-31T22:27:47.207+0800*:
|
||
|
||
收到
|
||
|
||
|
||
|
||
---
|
||
|
||
**liuyang** commented on *2023-04-04T09:22:01.118+0800*:
|
||
|
||
通过session record搜索server ip为目标IP，然后通过日志字段发现查看top 10的client ip是否能够满足需求？[~liuju]
|
||
|
||
|
||
|
||
---
|
||
|
||
**liuju** commented on *2023-05-04T16:01:28.997+0800*:
|
||
|
||
[~liuyang] 洋姐，你意思是像附件那种截图里的那样搜索嘛，搜索结果现场top 10 的结果是10个集合（source ips）和预期不符，想要的是top N client ip。
|
||
|
||
|
||
|
||
---
|
||
|
||
**liuju** commented on *2023-05-04T22:21:10.663+0800*:
|
||
|
||
根据虎哥提供的查询语句，查询4月份Destination IP为197.156.74.223/222出的top 10 client ip 全是10.x.x.x私网ip,后根据岱杰更新的语句，可以查询4月份Destination IP为197.156.74.223/222排除了私网IP之后的top 10 client ip和count ，目前查询的结果已上传展示在了附件中。
|
||
|
||
执行的更新后查询语句如下：
|
||
|
||
1、
|
||
clickhouse-client -h 10.224.11.35 --port 9001 -m -u default -d tsg_galaxy_v3 --password ***
|
||
|
||
2、
|
||
|
||
select item, sum(count) as count from ( select arrayJoin(items) as item,count from ( select splitByString(',', source_ip_list) as items, count(*)as count from dos_event lt where destination_ip = '27.50.64.2' and notEmpty(source_ip_list) and start_time >= toUnixTimestamp(parseDateTimeBestEffort(toString('2023-03-10 00:00:00+03:00'))) AND start_time < toUnixTimestamp(parseDateTimeBestEffort(toString('2023-03-26 00:00:00+03:00'))) group by source_ip_list ) ) where (item not like('10.%') AND item not like('172.%') AND item not like('192.168.%')) group by item order by count desc limit 10;
|
||
|
||
|
||
|
||
---
|
||
|
||
**liuyang** commented on *2023-05-12T20:58:19.110+0800*:
|
||
|
||
[~liuju]直接在 session record中搜索，不是dos event中搜索
|
||
!screenshot-1.png|thumbnail!
|
||
|
||
|
||
|
||
---
|
||
|
||
|
||
|
||
## Attachments
|
||
|
||
**36666/Critical+Dos+Events.docx**
|
||
|
||
---
|
||
|
||
**37679/image-2023-05-04-11-00-42-114.png**
|
||
|
||
---
|
||
|
||
**37923/screenshot-1.png**
|
||
|
||
---
|
||
|
||
**36665/微信图片_20230327212759.png**
|
||
|
||
---
|
||
|
||
**37681/微信图片_20230504171242.png**
|
||
|
||
---
|
||
|
||
**37682/微信图片_20230504171250.png**
|
||
|
||
---
|
||
|
||
**37683/微信图片_20230504171552.png**
|
||
|
||
---
|
||
|
||
**37684/微信图片_20230504171559.png**
|
||
|
||
---
|
||
|