1. 暂时去除AlertMessage.java中的implements以便于SpringBoot构建成功
2. 修改project.conf.template为project_template.conf 3. 修改README.md,添加大量项目部署解释 4. 添加StateChangeService.java中关于状态模式的相关解释
This commit is contained in:
1
.gitignore
vendored
1
.gitignore
vendored
@@ -7,6 +7,7 @@ build/
|
|||||||
*.gz
|
*.gz
|
||||||
!**/src/main/**/build/
|
!**/src/main/**/build/
|
||||||
!**/src/test/**/build/
|
!**/src/test/**/build/
|
||||||
|
docker-ubuntu/protection_ubuntu.tar
|
||||||
|
|
||||||
### STS ###
|
### STS ###
|
||||||
.apt_generated
|
.apt_generated
|
||||||
|
|||||||
75
README.md
75
README.md
@@ -8,13 +8,80 @@
|
|||||||
|
|
||||||
所有业务的配置,包括切面、拦截器、中间件、消息处理等配置全部放置于此处。
|
所有业务的配置,包括切面、拦截器、中间件、消息处理等配置全部放置于此处。
|
||||||
任何带@Configuration注解的配置均放置于此
|
任何带@Configuration注解的配置均放置于此
|
||||||
|
1. entity文件夹:均为下述业务模块的实体类
|
||||||
|
2. exception文件夹:用于存放自定义类和全局异常处理配置类
|
||||||
|
3. response文件夹:用于存放返回配置类和审计类
|
||||||
|
4. satoken文件夹:用于存放用户登录相关配置类
|
||||||
|
5. swagger文件夹:用于存放swagger文档配置类
|
||||||
|
6. threadpool文件夹:用于存放线程池配置
|
||||||
|
7. utils文件夹:用于存放其他类型配置
|
||||||
|
|
||||||
### server
|
### Server
|
||||||
|
|
||||||
server主要存放所有的业务逻辑以及Mapper接口。业务逻辑和Mapper需要按照业务进行划分进行分开放置
|
server主要存放所有的业务逻辑以及Mapper接口。业务逻辑和Mapper需要按照业务进行划分进行分开放置。
|
||||||
|
|
||||||
---
|
目前包含的业务有:
|
||||||
|
1. alertmessage:告警xx模块,处理告警相关信息
|
||||||
|
2. command:指令模块,处理下/发/指令
|
||||||
|
3. defense:防御模块,包括fy对象、fy策略模板两个子模块,用于处理fy对象、fy策略模板信息
|
||||||
|
4. log:日志模块,目前无文件
|
||||||
|
5. rule:规则模块,用于处理静态/动态规则
|
||||||
|
6. task:任务模块,用于处理任务相关信息
|
||||||
|
7. user:用户模块
|
||||||
|
8. whitelist:白名单模块,用于处理白名单配置
|
||||||
|
|
||||||
|
### Resources
|
||||||
除了项目文件夹外,还有静态资源**resources**部分,主要存放**config**、**mappers**和静态网页资源以及下载文件。
|
除了项目文件夹外,还有静态资源**resources**部分,主要存放**config**、**mappers**和静态网页资源以及下载文件。
|
||||||
|
|
||||||
**mappers**文件夹主要存放所有的Mybatis Mappers,以便于MyBatis进行扫描。
|
**mappers**文件夹主要存放所有的Mybatis Mappers,以便于MyBatis进行扫描。
|
||||||
**config**文件夹目前仅存放application.yml文件,用于进行SpringBoot和MyBatis基础配置。
|
|
||||||
|
**config**文件夹存放包括
|
||||||
|
1. ```application.yml```
|
||||||
|
2. ```application-dev.yml```
|
||||||
|
3. ```application-test.yml```
|
||||||
|
4. ```application-prod.yml```
|
||||||
|
|
||||||
|
等文件, 用于进行SpringBoot和MyBatis基础配置以及方便进行配置切换。
|
||||||
|
|
||||||
|
**nginx**文件夹包括了所有的前端界面和Nginx配置,其中:
|
||||||
|
1. nginx.conf用于进行Nginx服务器基础配置,关键设置为```include /root/nginx/configs/project.conf```;
|
||||||
|
2. project.conf.template文件是project.conf的模板文件,其可以利用环境变量以及envsubst指令生成project.conf文件(详细代码位于```start_nginx.sh```文件)
|
||||||
|
3. defense文件夹是前端界面文件夹
|
||||||
|
|
||||||
|
### Docker相关配置
|
||||||
|
在docker-ubuntu文件夹中存放的是nginx和springboot的Dockerfile中的```protection_ubuntu```镜像的制作方法,
|
||||||
|
如果想要重新制作```protection_ubuntu.tar```文件,请重新运行docker-ubuntu文件夹中的```Dockerfile```以制作基础ubuntu镜像。可以使用如下指令制作:
|
||||||
|
```shell
|
||||||
|
docker build -t protection_ubuntu .
|
||||||
|
```
|
||||||
|
同时,在docker部署之前,请执行以下指令以联网获取GRADLE相关资源依赖以便进行断网部署:
|
||||||
|
```shell
|
||||||
|
export set GRADLE_USER_HOME=./gradle
|
||||||
|
./gradlew build && ./graldew clean
|
||||||
|
```
|
||||||
|
在docker-nginx中存放的是nginx前端服务镜像的制作方法,全程理论可以做到断网部署,具体制作方法请检查```Dockerfile```以及```start_nginx.sh```
|
||||||
|
|
||||||
|
在docker-springboot中存放的是springboot后端服务镜像的制作方法,全程理论可以做到断网部署,具体制作方法请检查```Dockerfile```以及```start_springboot.sh```
|
||||||
|
|
||||||
|
如果想要制作前后端Docker镜像并开启docker容器,请使用```load_nginx.sh```和```load_springboot.sh```两个脚本(```load_springboot_multi.sh```仅用于测试Nginx服务器负载均衡用)
|
||||||
|
|
||||||
|
### 数据库相关配置
|
||||||
|
|
||||||
|
目前该项目使用两种数据库:Doris和MySQL,导入Doris和MySQL表结构可使用如下语句:
|
||||||
|
```shell
|
||||||
|
# 导入Doris数据库以及表结构,IP、端口、用户名均需要修改
|
||||||
|
mysql -u root -h 192.168.107.89 -P 9030 < realtime_protection/sql/doris-2024-1-23.sql -p
|
||||||
|
# 导入MySQL数据库以及表结构,IP、端口、用户名均需要修改
|
||||||
|
mysql -u root -h 192.168.107.89 -P 3306 < realtime_protection/sql/mysql-2024-1-23.sql -p
|
||||||
|
# 导入MySQL数据库以及表结构和表中所有数据,IP、端口、用户名均需要修改
|
||||||
|
mysql -u root -h 192.168.107.89 -P 3306 < realtime_protection/sql/mysql-2024-1-23-with-data.sql -p
|
||||||
|
```
|
||||||
|
如果需要导出Doris表结构和MySQL表结构以及数据,可使用如下语句:
|
||||||
|
```shell
|
||||||
|
# 导出MySQL表结构和数据
|
||||||
|
mysqldump -u root -h 192.168.107.89 -P 3306 --databases realtime_protection > mysql-2024-01-23-with-data.sql -p
|
||||||
|
# 导出MySQL表结构,不导出数据
|
||||||
|
mysqldump -u root -h 192.168.107.89 -P 3306 --databases realtime_protection -no-data > mysql-2024-01-23-without-data.sql -p
|
||||||
|
# 导出doris表结构
|
||||||
|
mysqldump -u root -h 192.168.107.89 -P 9030 --databases command --no-tablespaces -no-data > doris-2024-01-23.sql -p
|
||||||
|
```
|
||||||
@@ -3,10 +3,10 @@ LABEL authors="Endera"
|
|||||||
|
|
||||||
COPY realtime_protection realtime_protection
|
COPY realtime_protection realtime_protection
|
||||||
|
|
||||||
ENV SPRINGBOOT_FIRST_URL="localhost:8081" \
|
ENV SPRINGBOOT_FIRST_URL="172.24.100.53:8081" \
|
||||||
SPRINGBOOT_SECOND_URL="localhost:8082" \
|
SPRINGBOOT_SECOND_URL="172.24.100.53:8082" \
|
||||||
SPRINGBOOT_THIRD_URL="localhost:8083" \
|
SPRINGBOOT_THIRD_URL="172.24.100.53:8083" \
|
||||||
SPRINGBOOT_FOURTH_URL="localhost:8084" \
|
SPRINGBOOT_FOURTH_URL="172.24.100.53:8084" \
|
||||||
NGINX_PORT=8080 \
|
NGINX_PORT=8080 \
|
||||||
NGINX_SERVER_NAME="localhost"
|
NGINX_SERVER_NAME="localhost"
|
||||||
|
|
||||||
|
|||||||
@@ -7,7 +7,7 @@ export set PROTECTION_NGINX_DIR="$PROTECTION_DIR/src/main/resources/nginx"
|
|||||||
export set PROJECT_NGINX_DIR="$BASEDIR/nginx"
|
export set PROJECT_NGINX_DIR="$BASEDIR/nginx"
|
||||||
export set GLOBAL_NGINX_DIR="/etc/nginx"
|
export set GLOBAL_NGINX_DIR="/etc/nginx"
|
||||||
|
|
||||||
envsubst '${SPRINGBOOT_FIRST_URL} ${SPRINGBOOT_SECOND_URL} ${SPRINGBOOT_THIRD_URL} ${SPRINGBOOT_FOURTH_URL} ${NGINX_PORT} ${NGINX_SERVER_NAME}' < $PROTECTION_NGINX_DIR/configs/project.conf.template > $PROTECTION_NGINX_DIR/configs/project.conf
|
envsubst '${SPRINGBOOT_FIRST_URL} ${SPRINGBOOT_SECOND_URL} ${SPRINGBOOT_THIRD_URL} ${SPRINGBOOT_FOURTH_URL} ${NGINX_PORT} ${NGINX_SERVER_NAME}' < $PROTECTION_NGINX_DIR/configs/project_template.conf > $PROTECTION_NGINX_DIR/configs/project.conf
|
||||||
|
|
||||||
# 拷贝Nginx配置到指定地址
|
# 拷贝Nginx配置到指定地址
|
||||||
cp -r $PROTECTION_NGINX_DIR $BASEDIR
|
cp -r $PROTECTION_NGINX_DIR $BASEDIR
|
||||||
|
|||||||
@@ -10,10 +10,7 @@ rm -rf .gradle
|
|||||||
cd ..
|
cd ..
|
||||||
|
|
||||||
echo "============start loading springboot============"
|
echo "============start loading springboot============"
|
||||||
docker stop springboot_backend && docker rm springboot_backend
|
docker rm -f $(docker ps -a | grep springboot_backend | awk '{print $1}')
|
||||||
docker stop springboot_backend_1 && docker rm springboot_backend_1
|
|
||||||
docker stop springboot_backend_2 && docker rm springboot_backend_2
|
|
||||||
docker stop springboot_backend_3 && docker rm springboot_backend_3
|
|
||||||
docker image rm protection_springboot protection_ubuntu
|
docker image rm protection_springboot protection_ubuntu
|
||||||
echo "============old containers and images are cleared============"
|
echo "============old containers and images are cleared============"
|
||||||
|
|
||||||
@@ -26,4 +23,5 @@ echo "============built backend servers============"
|
|||||||
docker run -itd --name springboot_backend_1 -p 8081:8081 protection_springboot
|
docker run -itd --name springboot_backend_1 -p 8081:8081 protection_springboot
|
||||||
docker run -itd --name springboot_backend_2 -p 8082:8081 protection_springboot
|
docker run -itd --name springboot_backend_2 -p 8082:8081 protection_springboot
|
||||||
docker run -itd --name springboot_backend_3 -p 8083:8081 protection_springboot
|
docker run -itd --name springboot_backend_3 -p 8083:8081 protection_springboot
|
||||||
|
docker run -itd --name springboot_backend_4 -p 8084:8081 protection_springboot
|
||||||
echo "============started backend servers============"
|
echo "============started backend servers============"
|
||||||
@@ -3,11 +3,10 @@ package com.realtime.protection.configuration.entity.alert;
|
|||||||
|
|
||||||
import com.fasterxml.jackson.annotation.JsonProperty;
|
import com.fasterxml.jackson.annotation.JsonProperty;
|
||||||
import com.realtime.protection.configuration.entity.task.FiveTupleWithMask;
|
import com.realtime.protection.configuration.entity.task.FiveTupleWithMask;
|
||||||
import com.realtime.protection.configuration.kafka.KafkaMessage;
|
|
||||||
import lombok.Data;
|
import lombok.Data;
|
||||||
|
|
||||||
@Data
|
@Data
|
||||||
public class AlertMessage implements KafkaMessage {
|
public class AlertMessage {
|
||||||
|
|
||||||
@JsonProperty("task_id")
|
@JsonProperty("task_id")
|
||||||
private Long taskId;
|
private Long taskId;
|
||||||
|
|||||||
@@ -10,6 +10,7 @@ import java.util.Map;
|
|||||||
@Getter
|
@Getter
|
||||||
public enum StateEnum {
|
public enum StateEnum {
|
||||||
// 仅需修改此处即可将任务状态以及对应的State和Num进行对应
|
// 仅需修改此处即可将任务状态以及对应的State和Num进行对应
|
||||||
|
// 使用状态模式进行任务状态的管理和切换
|
||||||
PENDING(0, new PendingState()),
|
PENDING(0, new PendingState()),
|
||||||
GENERATING(1, new GeneratingState()),
|
GENERATING(1, new GeneratingState()),
|
||||||
RUNNING(2, new RunningState()),
|
RUNNING(2, new RunningState()),
|
||||||
|
|||||||
@@ -15,6 +15,9 @@ import org.springframework.stereotype.Service;
|
|||||||
import java.util.List;
|
import java.util.List;
|
||||||
import java.util.Objects;
|
import java.util.Objects;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 任务状态管理服务类
|
||||||
|
*/
|
||||||
@Service
|
@Service
|
||||||
@EnableScheduling
|
@EnableScheduling
|
||||||
@Slf4j
|
@Slf4j
|
||||||
@@ -27,6 +30,14 @@ public class StateChangeService {
|
|||||||
this.taskService = taskService;
|
this.taskService = taskService;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 切换任务状态
|
||||||
|
* @param stateNum 欲切换的任务状态编号(编号对应的状态请查看configuration/utils/enums/StateEnum类)
|
||||||
|
* @param taskId 任务ID
|
||||||
|
* @param inner 此接口是否为内部调用,如果为内部调用,则取消所有的状态检查
|
||||||
|
* @return 状态切换是否成功
|
||||||
|
* @throws DorisStartException Doris数据库异常
|
||||||
|
*/
|
||||||
@DSTransactional
|
@DSTransactional
|
||||||
public Boolean changeState(Integer stateNum, Long taskId, Boolean inner) throws DorisStartException {
|
public Boolean changeState(Integer stateNum, Long taskId, Boolean inner) throws DorisStartException {
|
||||||
Integer originalStateNum = taskService.queryTaskStatus(taskId);
|
Integer originalStateNum = taskService.queryTaskStatus(taskId);
|
||||||
@@ -49,6 +60,7 @@ public class StateChangeService {
|
|||||||
return true;
|
return true;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// 切换到states文件夹中查看每一个状态的handle函数
|
||||||
if (!originalState.handle(newState, commandService, taskService, taskId)) {
|
if (!originalState.handle(newState, commandService, taskService, taskId)) {
|
||||||
return false;
|
return false;
|
||||||
}
|
}
|
||||||
@@ -79,6 +91,9 @@ public class StateChangeService {
|
|||||||
return !Objects.equals(originalState, StateEnum.GENERATING.getState());
|
return !Objects.equals(originalState, StateEnum.GENERATING.getState());
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 将任务切换为结束状态
|
||||||
|
*/
|
||||||
@Scheduled(cron = "0 0/10 * * * ?")
|
@Scheduled(cron = "0 0/10 * * * ?")
|
||||||
@Async
|
@Async
|
||||||
protected void finishTasks() {
|
protected void finishTasks() {
|
||||||
|
|||||||
@@ -1,39 +0,0 @@
|
|||||||
upstream server_list {
|
|
||||||
least_conn;
|
|
||||||
server 192.168.107.89:8081;
|
|
||||||
}
|
|
||||||
server {
|
|
||||||
listen 8080;
|
|
||||||
server_name localhost;
|
|
||||||
location / {
|
|
||||||
root /root/nginx/defense;
|
|
||||||
index index.html;
|
|
||||||
try_files $uri $uri/ @router;
|
|
||||||
}
|
|
||||||
|
|
||||||
location @router {
|
|
||||||
rewrite ^.*$ /index.html last;
|
|
||||||
}
|
|
||||||
|
|
||||||
location /api/ {
|
|
||||||
add_header 'Access-Control-Allow-Origin' $http_origin;
|
|
||||||
add_header 'Access-Control-Allow-Credentials' 'true';
|
|
||||||
add_header 'Access-Control-Allow-Methods' 'GET, POST, DELETE, OPTIONS';
|
|
||||||
add_header 'Access-Control-Allow-Headers' 'DNT,Authorization,Accept,Origin,Keep-Alive,User-Agent,X-Mx-ReqToken,X-Data-Type,X-Auth-Token,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range';
|
|
||||||
add_header 'Access-Control-Expose-Headers' 'Content-Length,Content-Range';
|
|
||||||
if ($request_method = 'OPTIONS') {
|
|
||||||
add_header 'Access-Control-Max-Age' 1728000;
|
|
||||||
add_header 'Content-Type' 'text/plain; charset=utf-8';
|
|
||||||
add_header 'Content-Length' 0;
|
|
||||||
return 204;
|
|
||||||
}
|
|
||||||
|
|
||||||
proxy_set_header Host $host:$server_port;
|
|
||||||
proxy_set_header X-Real-IP $remote_addr;
|
|
||||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
|
||||||
proxy_set_header X-Forwarded-Proto $scheme;
|
|
||||||
proxy_connect_timeout 5;
|
|
||||||
|
|
||||||
proxy_pass http://server_list/api/v1/;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -58,7 +58,7 @@ http {
|
|||||||
|
|
||||||
# include /etc/nginx/conf.d/*.conf;
|
# include /etc/nginx/conf.d/*.conf;
|
||||||
# include /etc/nginx/sites-enabled/*;
|
# include /etc/nginx/sites-enabled/*;
|
||||||
include /root/nginx/configs/*.conf;
|
include /root/nginx/configs/project.conf;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user