gfwleak/tango-tsg-master
Archived
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Squashed commit of the following:

Browse Source 
commit 91574444c3
Author: liuxueli <liuxueli@iie.ac.cn>
Date:   Tue Dec 3 11:44:16 2019 +0800

    公共字段中的通信市时长单位是：ms；转换单位
    KNI发日志需等待TFE结束，需提前构造streaminfo日志字段，调整包数字节数统计位置
    细化调整日志

commit ff053e3e2c
Author: liuxueli <liuxueli@iie.ac.cn>
Date:   Wed Nov 27 11:38:35 2019 +0800

    增加调试日志信息

commit 2d1a530165
Author: liuxueli <liuxueli@iie.ac.cn>
Date:   Wed Nov 27 11:27:06 2019 +0800

    使用前置声明结构体，修改接口函数原型

commit 27f0cfc91a
Author: liuxueli <liuxueli@iie.ac.cn>
Date:   Wed Nov 27 10:33:48 2019 +0800

    提供TLD_append_streaminfo函数接口供KNI使用
    修改配置文件

commit ed0b17a49d
Merge: 132d807 04963fe
Author: 崔一鸣 <cuiyiming@iie.ac.cn>
Date:   Tue Nov 26 17:44:28 2019 +0800

    Merge branch 'feature-kni' into 'develop'

    增加http解析部分

    See merge request tango/tsg_master!3

commit 04963fe1bf
Author: 崔一鸣 <cuiyiming@iie.ac.cn>
Date:   Tue Nov 26 17:41:43 2019 +0800

    增加http解析部分

commit 132d807af1
Author: liuxueli <liuxueli@iie.ac.cn>
Date:   Tue Nov 26 16:58:54 2019 +0800

    调整调试日志内容

commit 8cfc6b32fd
Author: liuxueli <liuxueli@iie.ac.cn>
Date:   Mon Nov 25 19:30:08 2019 +0800

    增加调试日志信息

commit 4aef0628e5
Author: liuxueli <liuxueli@iie.ac.cn>
Date:   Mon Nov 25 18:36:48 2019 +0800

    增加调试信息

commit 7c9dabdab5
Author: liuxueli <liuxueli@iie.ac.cn>
Date:   Mon Nov 25 10:51:37 2019 +0800

    矫正tableinfo配置项

commit 34ec2059ea
Merge: 27cb2ad 2b96c05
Author: liuxueli <liuxueli@iie.ac.cn>
Date:   Mon Nov 25 10:00:18 2019 +0800

    Merge branch 'develop' of https://git.mesalab.cn/tango/tsg_master into develop

commit 27cb2ad3b2
Author: liuxueli <liuxueli@iie.ac.cn>
Date:   Thu Nov 21 19:21:08 2019 +0800

    获取包数、字节数与sapp头文件不一致，导致统计信息颠倒
    删除从vxlan里获取信息

commit 2b96c050d5
Merge: 0a11db6 8d954e4
Author: 崔一鸣 <cuiyiming@iie.ac.cn>
Date:   Thu Nov 21 10:22:42 2019 +0800

    Merge branch 'feature-kni' into 'develop'

    修改proto初始值为PROTO_UNKONWN

    See merge request tango/tsg_master!2

commit 46a233b782
Author: liuxueli <liuxueli@iie.ac.cn>
Date:   Thu Nov 21 10:18:08 2019 +0800

    KNI发送日志的线程不是sapp申请的，所以不能使用dictator_malloc申请内存
    增加user_region日志字段

commit 8d954e4cc5
Author: 崔一鸣 <cuiyiming@iie.ac.cn>
Date:   Wed Nov 20 19:55:59 2019 +0800

    修改proto初始值为PROTO_UNKONWN

commit 0a11db6d2e
Author: liuxueli <liuxueli@iie.ac.cn>
Date:   Wed Nov 20 18:02:54 2019 +0800

    增加总控配置文件

commit 5c04ba23aa
Author: liuxueli <liuxueli@iie.ac.cn>
Date:   Wed Nov 20 18:00:35 2019 +0800

    SSL_SNI和HTTP_HOST的deny动作由总控负责处理
    修复获取全局流ID时的错误

commit 291c05ea79
Author: liuxueli <liuxueli@iie.ac.cn>
Date:   Wed Nov 20 16:40:19 2019 +0800

    增加获取全局流ID的接口
    调整总控注册为TCP入口

commit 28da97b53b
Author: liuxueli <liuxueli@iie.ac.cn>
Date:   Wed Nov 20 14:45:41 2019 +0800

    支持平台提供的APP_STATE_KILL_OTHER
    修复BUG，天剑流标签时未对协议进行赋值，导致tfe颁发证书失败

commit 7912ec54b2
Author: liuxueli <liuxueli@iie.ac.cn>
Date:   Tue Nov 19 18:41:31 2019 +0800

    修复BUG，命中拦截策略时未对类型进行赋值，导致KNI获取不到命中结果
    添加总控配置文件
    添加FS2统计日志
    添加错误日志
    构造适用sapp4.0的rpm

commit 484feaed94
Author: liuxueli <liuxueli@iie.ac.cn>
Date:   Tue Nov 19 13:39:37 2019 +0800

    增加获取deny结果规则的接口
    修复自测试过程中遇到的BUG

commit 5165173025
Merge: 186e591 6b6cbef
Author: 崔一鸣 <cuiyiming@iie.ac.cn>
Date:   Mon Nov 18 15:46:03 2019 +0800

    Merge branch 'feature-fast-path' into feature-kni

commit 6b6cbeffa1
Author: liuxueli <liuxueli@iie.ac.cn>
Date:   Mon Nov 18 14:50:32 2019 +0800

    修复初始化错误

commit e7532d6280
Author: liuxueli <liuxueli@iie.ac.cn>
Date:   Mon Nov 18 13:33:29 2019 +0800

    添加遗漏头文件tsg_entry.h

commit f33d11c50b
Author: liuxueli <liuxueli@iie.ac.cn>
Date:   Mon Nov 18 13:29:28 2019 +0800

    修复编译错误

commit 88d9368cee
Author: liuxueli <liuxueli@iie.ac.cn>
Date:   Mon Nov 18 13:25:38 2019 +0800

    合并头文件，删除无用头文件

commit 382ca0a793
Author: 刘学利 <liuxueli@iie.ac.cn>
Date:   Mon Nov 18 13:21:25 2019 +0800

    Delete tsg_types.h

commit ab58f918e5
Author: 刘学利 <liuxueli@iie.ac.cn>
Date:   Mon Nov 18 13:21:05 2019 +0800

    Delete tsg_log_id.h

commit 3b13075701
Author: 刘学利 <liuxueli@iie.ac.cn>
Date:   Mon Nov 18 13:20:57 2019 +0800

    Delete tsg_entry.h

commit 186e591dbf
Merge: 39f82ed 91c511c
Author: 崔一鸣 <cuiyiming@iie.ac.cn>
Date:   Fri Nov 15 21:22:44 2019 +0800

    Merge branch 'master' into feature-kni

commit 39f82ed3f7
Author: 崔一鸣 <cuiyiming@iie.ac.cn>
Date:   Fri Nov 15 21:22:39 2019 +0800

    接口适配

commit 8913a7d22e
Author: liuxueli <liuxueli@iie.ac.cn>
Date:   Fri Nov 15 19:29:54 2019 +0800

    规则排序功能
    统一发送日志功能

commit 1cb2358fea
Merge: c76bc55 bd65acc
Author: 崔一鸣 <cuiyiming@iie.ac.cn>
Date:   Thu Nov 14 15:06:35 2019 +0800

    合并master修改

commit c76bc5534d
Author: 崔一鸣 <cuiyiming@iie.ac.cn>
Date:   Wed Nov 13 19:56:19 2019 +0800

    接口增加protocol参数

commit db7282dab7
Author: 崔一鸣 <cuiyiming@iie.ac.cn>
Date:   Wed Nov 13 19:24:22 2019 +0800

    扫描接口增加domain

commit 56fadd73b2
Merge: f9aee05 9e738ae
Author: 崔一鸣 <cuiyiming@iie.ac.cn>
Date:   Wed Nov 13 19:08:55 2019 +0800

    Merge branch 'master' into feature-kni

commit f9aee05bcc
Author: 崔一鸣 <cuiyiming@iie.ac.cn>
Date:   Wed Nov 13 16:17:59 2019 +0800

    修改ssl_utils文件名

commit 452a08790c
Merge: 3abe8a2 c1ffc53
Author: 崔一鸣 <cuiyiming@iie.ac.cn>
Date:   Wed Nov 13 16:12:14 2019 +0800

    Merge branch 'master' into feature-kni

commit 3abe8a2fc5
Author: 崔一鸣 <cuiyiming@iie.ac.cn>
Date:   Wed Nov 13 16:11:38 2019 +0800

    添加kni模块扫描部分
This commit is contained in:
杨威
2019-12-09 18:58:05 +08:00
parent 91c511c3ed
commit 64c1dcb159
22 changed files with 2184 additions and 357 deletions
Download Patch File Download Diff File Expand all files Collapse all files

39
bin/main.conf Normal file
View File

@@ -0,0 +1,39 @@
[MAAT]
MAAT_MODE=1
#EFFECTIVE_FLAG=
STAT_SWITCH=1
PERF_SWITCH=1
TABLE_INFO=tsgconf/tsg_tableinfo.conf
STAT_FILE=tsg_maat.status
EFFECT_INTERVAL_S=1
REDIS_IP=127.0.0.1
REDIS_PORT_NUM=10
REDIS_PORT=6380
REDIS_INDEX=2
JSON_CFG_FILE=tsgconf/tsg_maat.json
INC_CFG_DIR=tsgrule/inc/index/
FULL_CFG_DIR=tsgrule/full/index/
IP_ADDR_TABLE=TSG_OBJ_IP_ADDR
SUBSCRIBER_ID_TABLE=TSG_OBJ_SUBSCRIBER_ID
[TSG_LOG]
MODE=1
NIC_NAME=eth1
MAX_SERVICE=0
LOG_LEVEL=10
LOG_PATH=./tsglog/tsglog
BROKER_LIST=127.0.0.1:9092
COMMON_FIELD_FILE=tsgconf/tsg_log_field.conf
[FIELD_STAT]
CYCLE=3
TELEGRAF_PORT=8125
TELEGRAF_IP=127.0.0.1
OUTPUT_PATH=./tsg_stat.log
APP_NAME=tsg_master
[SYSTEM]
LOG_LEVEL=10
LOG_PATH=./tsglog/tsg_master
POLICY_PRIORITY_LABEL=POLICY_PRIORITY

41
bin/tsg_log_field.conf Normal file
View File

@@ -0,0 +1,41 @@
#TYPE：1:UCHAR,2:USHORT,3:ULONG,4:ULOG,5:USTRING,6:FILE,7:UBASE64,8:PACKET
#TYPE TOPIC SERVICE
TOPIC SECURITY-EVENT-LOG 0
#TYPE FIELD VALUE
LONG common_policy_id 1
LONG common_service 2
LONG common_action 3
LONG common_start_time 4
LONG common_end_time 5
STRING common_l4_protocol 6
LONG common_address_type 7
STRING common_server_ip 8
STRING common_client_ip 9
LONG common_server_port 10
LONG common_client_port 11
LONG common_stream_dir 12
STRING common_address_list 13
LONG common_entrance_id 14
LONG common_device_id 15
LONG common_link_id 16
STRING common_isp 17
LONG common_encapsulation 18
LONG common_direction 19
STRING common_sled_ip 20
STRING common_user_tags 21
STRING common_user_region 22
STRING common_app_label 23
LONG common_app_id 24
LONG common_protocol_id 25
LONG common_c2s_pkt_num 26
LONG common_s2c_pkt_num 27
LONG common_c2s_byte_num 28
LONG common_s2c_byte_num 29
LONG common_con_duration_ms 30
LONG common_has_dup_traffic 31
STRING common_stream_error 32
STRING common_stream_trace_id 33
STRING common_schema_type 34
STRING http_host 35
STRING ssl_sni 36

84
bin/tsg_maat.json Normal file
View File

@@ -0,0 +1,84 @@
{
"compile_table": "TSG_SECURITY_COMPILE",
"group_table": "POLICY_OBJECT",
"rules": [
{
"compile_id": 1,
"service": 0,
"action": 16,
"do_blacklist": 0,
"do_log": 1,
"effective_rage": 0,
"user_region": "anything",
"is_valid": "yes",
"groups": [
{
"group_name": "group_1",
"regions": [
{
"table_name": "TSG_OBJ_IP_ADDR",
"table_type": "ip",
"table_content": {
"addr_type": "ipv4",
"src_ip": "61.135.169.125",
"mask_src_ip": "255.255.255.255",
"src_port": "80",
"mask_src_port": "65535",
"dst_ip": "192.168.41.228",
"mask_dst_ip": "255.255.255.255",
"dst_port": "0",
"mask_dst_port": "65535",
"protocol": 6,
"direction": "double"
}
}
]
}
]
},
{
"compile_id": 2,
"service": 0,
"action": 128,
"do_blacklist": 0,
"do_log": 1,
"effective_rage": 0,
"user_region": "anything",
"is_valid": "yes",
"groups": [
{
"group_name": "FQDN_SNI",
"regions": [
{
"table_name": "TSG_OBJ_FQDN",
"table_type": "expr",
"table_content": {
"keywords": "baidu.com",
"expr_type": "and",
"match_method": "sub",
"format": "uncase plain"
}
}
]
}
]
},
{
"compile_id": 3,
"service": 0,
"action": 128,
"do_blacklist": 0,
"do_log": 1,
"effective_rage": 0,
"user_region": "Virtual",
"is_valid": "yes",
"groups": [
{
"group_name":"FQDN_SNI",
"virtual_table":"TSG_FIELD_SSL_SNI",
"not_flag" : 0
}
]
}
]
}

40
bin/tsg_maat_ip_deny.json Normal file
View File

@@ -0,0 +1,40 @@
{
"compile_table": "TSG_SECURITY_COMPILE",
"group_table": "POLICY_OBJECT",
"rules": [
{
"compile_id": 1,
"service": 0,
"action": 16,
"do_blacklist": 0,
"do_log": 1,
"effective_rage": 0,
"user_region": "anything",
"is_valid": "yes",
"groups": [
{
"group_name": "group_1",
"regions": [
{
"table_name": "TSG_OBJ_IP_ADDR",
"table_type": "ip",
"table_content": {
"addr_type": "ipv4",
"src_ip": "117.18.237.29",
"mask_src_ip": "255.255.255.255",
"src_port": "80",
"mask_src_port": "65535",
"dst_ip": "192.168.41.228",
"mask_dst_ip": "255.255.255.255",
"dst_port": "0",
"mask_dst_port": "65535",
"protocol": 6,
"direction": "double"
}
}
]
}
]
}
]
}

9
bin/tsg_master.inf Normal file
View File

@@ -0,0 +1,9 @@
[PLUGINFO]
PLUGNAME=TSG_MASTER
SO_PATH=./plug/platform/tsg_master/tsg_master.so
INIT_FUNC=TSG_MASTER_INIT
DESTROY_FUNC=TSG_MASTER_UNLOAD
[TCP]
FUNC_FLAG=ALL
FUNC_NAME=TSG_MASTER_TCP_ENTRY

40
bin/tsg_tableinfo.conf Normal file
View File

@@ -0,0 +1,40 @@
#each collumn seperate with '\t'
#id (0~65535)
#name string
#type one of ip,expr,expr_plus,digest,intval,compile or plugin
#src_charset one of GBK,BIG5,UNICODE,UTF8
#dst_charset combined by GBK,BIG5,UNICODE,UTF8,seperate with '/'
#do_merege yes or no
#cross cache 0~max
#quickswitch quickon or quick off
#id name type src_charset dst_charset do_merge cross_cache quickswitch
0 TSG_SECURITY_COMPILE compile escape --
1 GROUP_COMPILE_RELATION group UTF8 UTF8 no 0
2 TSG_OBJ_IP_ADDR ip_plus UTF8 UTF8 no 0
3 TSG_OBJ_SUBSCRIBER_ID expr UTF8 UTF8 no 0
4 TSG_OBJ_ACCOUNT expr UTF8 UTF8 no 0
5 TSG_OBJ_URL expr UTF8 UTF8/GBK no 0
6 TSG_OBJ_FQDN expr UTF8 UTF8 no 0
7 TSG_OBJ_KEYWORDS expr UTF8 UTF8 no 0
8 TSG_OBJ_HTTP_SIGNATURE expr_plus UTF8 UTF8/GBK no 0
9 TSG_FIELD_HTTP_HOST virtual TSG_OBJ_FQDN --
10 TSG_FIELD_HTTP_URL virtual TSG_OBJ_URL --
11 TSG_FIELD_HTTP_REQ_HDR virtual TSG_OBJ_HTTP_SIGNATURE --
12 TSG_FIELD_HTTP_RES_HDR virtual TSG_OBJ_HTTP_SIGNATURE --
13 TSG_FIELD_HTTP_REQ_CONTENT virtual TSG_OBJ_KEYWORDS --
14 TSG_FIELD_HTTP_RES_CONTENT virtual TSG_OBJ_KEYWORDS --
15 TSG_FIELD_SSL_SNI virtual TSG_OBJ_FQDN --
16 TSG_FIELD_SSL_CN virtual TSG_OBJ_FQDN --
17 TSG_FIELD_SSL_SAN virtual TSG_OBJ_FQDN --
18 TSG_FIELD_DNS_QNAME virtual TSG_OBJ_FQDN --
19 TSG_FIELD_MAIL_ACCOUNT virtual TSG_OBJ_ACCOUNT --
20 TSG_FIELD_MAIL_FROM virtual TSG_OBJ_ACCOUNT --
21 TSG_FIELD_MAIL_TO virtual TSG_OBJ_ACCOUNT --
22 TSG_FIELD_MAIL_SUBJECT virtual TSG_OBJ_KEYWORDS --
23 TSG_FIELD_MAIL_CONTENT virtual TSG_OBJ_KEYWORDS --
24 TSG_FIELD_MAIL_ATT_NAME virtual TSG_OBJ_KEYWORDS --
25 TSG_FIELD_MAIL_ATT_CONTENT virtual TSG_OBJ_KEYWORDS --
26 TSG_FIELD_FTP_URI virtual TSG_OBJ_URL --
27 TSG_FIELD_FTP_CONTENT virtual TSG_OBJ_KEYWORDS --
28 TSG_FIELD_FTP_ACCOUNT virtual TSG_OBJ_ACCOUNT --