gfwleak/galaxy-platform-galaxy-troubleshooting-api
Archived
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update README.md

Browse Source
This commit is contained in:
窦凤虎
2024-04-26 03:39:50 +00:00
parent f5d1b9b8e9
commit 44b31e9ec8
1 changed files with 46 additions and 28 deletions
Download Patch File Download Diff File Expand all files Collapse all files

74
24.05/tsg_olap_e2e_test/README.md
View File

@@ -2,7 +2,11 @@
## 概述
TSG OLAP的数据摄入分为三种类型：Logs、Metrics 和 File Chunks。为确保能够正确处理并写入相应存储系统，需要支持端到端业务自检。
TSG OLAP的数据摄入分为三种类型：Logs、Metrics 和 File Chunks。为确保能够正确处理并写入相应存储系统，需要支持端到端业务自检。支持部署方式：
- 集中式部署
- 多数据中心部署
![End-to-End Workflow](images/e2e-test-flow-figure.png)
## 环境依赖
@@ -13,29 +17,45 @@ TSG OLAP的数据摄入分为三种类型：Logs、Metrics 和 File Chunks。为
## 使用方法
### 配置QGW API
- 修改 `enviroment.json`，配置QGW的IP地址和端口 (首次安装)
### 配置Newman CLI enviroment.json
```json
{
"key": "qgw_ip",
"value": "192.168.44.30",
"type": "default",
"enabled": true
},
{
"key": "qgw_port",
"value": "9999",
"type": "default",
"enabled": true
}
"key": "qgw_ip",
"value": "192.168.44.30",
"type": "default",
"enabled": true
},
{
"key": "qgw_port",
"value": "9999",
"type": "default",
"enabled": true
},
{
"key": "hos_ip",
"value": "192.168.44.30",
"type": "default",
"enabled": true
},
{
"key": "hos_port",
"value": "9098",
"type": "default",
"enabled": true
},
{
"key": "hos_token",
"value": "xxxxxx",
"type": "default",
"enabled": true
}
```
### 配置File Chunks 生成工具
- 修改`config.properties`, 增加Kafka Topic名称和连接地址
- 修改`config.properties`, 增加Kafka访问地址
```props
kafka.server=192.168.41.29:9092
kafka.topic=TRAFFIC-FILE-STREAM-RECORD
kafka.server=192.168.41.29:9092
```
### 写入测试集至Kafka
@@ -68,15 +88,17 @@ kafka-operation.sh producer STATISTICS-RULE-METRIC < ./datasets/metrics/statisti
```shell
cd file-chunk-generator/
java -jar file-chunk-generator.jar -n 123e4567-e89b-12d3-a456-426614174001 --file_type pcapng
java -jar file-chunk-generator.jar --topic TRAFFIC-FILE-STREAM-RECORD -n 123e4567-e89b-12d3-a456-426614174001 --file_type traffic_pcapng
java -jar file-chunk-generator.jar -n 123e4567-e89b-12d3-a456-426614174002 --file_type html
java -jar file-chunk-generator.jar --topic TRAFFIC-FILE-STREAM-RECORD -n 123e4567-e89b-12d3-a456-426614174002 --file_type html
java -jar file-chunk-generator.jar -n 123e4567-e89b-12d3-a456-426614174003 --file_type html
java -jar file-chunk-generator.jar --topic TRAFFIC-FILE-STREAM-RECORD -n 123e4567-e89b-12d3-a456-426614174003 --file_type html
java -jar file-chunk-generator.jar -n 123e4567-e89b-12d3-a456-426614174004 --file_type eml
java -jar file-chunk-generator.jar --topic TRAFFIC-FILE-STREAM-RECORD -n 123e4567-e89b-12d3-a456-426614174004 --file_type eml
java -jar file-chunk-generator.jar -n 123e4567-e89b-12d3-a456-426614174005 --file_type pcapng
java -jar file-chunk-generator.jar --topic TRAFFIC-FILE-STREAM-RECORD -n 123e4567-e89b-12d3-a456-426614174005 --file_type traffic_pcapng
java -jar file-chunk-generator.jar --topic TROUBLESHOOTING-FILE-STREAM-RECORD -n 123e4567-e89b-12d3-a456-426614174006 --file_type troubleshooting_pcapng
```
@@ -93,13 +115,9 @@ newman run ./collection.json -n 1 --delay-request 500 -e ./environment.json --ig
#Logs进行故障诊断，输出报告json格式，自动存储newman目录
newman run ./collection.json -n 1 --delay-request 500 -e ./environment.json --ignore-redirects --folder logs -r cli,json
#清除测试数据（暂支持文件删除）
#清除测试数据（暂支持对文件的删除）
newman run ./collection.json -n 1 --delay-request 500 -e ./environment.json --ignore-redirects --folder clear_test_data -r emojitrain
```