1. 暂时去除AlertMessage.java中的implements以便于SpringBoot构建成功

2. 修改project.conf.template为project_template.conf
3. 修改README.md,添加大量项目部署解释
4. 添加StateChangeService.java中关于状态模式的相关解释
This commit is contained in:
EnderByEndera
2024-01-25 19:55:23 +08:00
parent 37b9e1f146
commit e48f837b64
11 changed files with 97 additions and 55 deletions

1
.gitignore vendored
View File

@@ -7,6 +7,7 @@ build/
*.gz *.gz
!**/src/main/**/build/ !**/src/main/**/build/
!**/src/test/**/build/ !**/src/test/**/build/
docker-ubuntu/protection_ubuntu.tar
### STS ### ### STS ###
.apt_generated .apt_generated

View File

@@ -8,13 +8,80 @@
所有业务的配置,包括切面、拦截器、中间件、消息处理等配置全部放置于此处。 所有业务的配置,包括切面、拦截器、中间件、消息处理等配置全部放置于此处。
任何带@Configuration注解的配置均放置于此 任何带@Configuration注解的配置均放置于此
1. entity文件夹均为下述业务模块的实体类
2. exception文件夹用于存放自定义类和全局异常处理配置类
3. response文件夹用于存放返回配置类和审计类
4. satoken文件夹用于存放用户登录相关配置类
5. swagger文件夹用于存放swagger文档配置类
6. threadpool文件夹用于存放线程池配置
7. utils文件夹用于存放其他类型配置
### server ### Server
server主要存放所有的业务逻辑以及Mapper接口。业务逻辑和Mapper需要按照业务进行划分进行分开放置 server主要存放所有的业务逻辑以及Mapper接口。业务逻辑和Mapper需要按照业务进行划分进行分开放置
--- 目前包含的业务有:
1. alertmessage告警xx模块处理告警相关信息
2. command指令模块处理下/发/指令
3. defense防御模块包括fy对象、fy策略模板两个子模块用于处理fy对象、fy策略模板信息
4. log日志模块目前无文件
5. rule规则模块用于处理静态/动态规则
6. task任务模块用于处理任务相关信息
7. user用户模块
8. whitelist白名单模块用于处理白名单配置
### Resources
除了项目文件夹外,还有静态资源**resources**部分,主要存放**config**、**mappers**和静态网页资源以及下载文件。 除了项目文件夹外,还有静态资源**resources**部分,主要存放**config**、**mappers**和静态网页资源以及下载文件。
**mappers**文件夹主要存放所有的Mybatis Mappers以便于MyBatis进行扫描。 **mappers**文件夹主要存放所有的Mybatis Mappers以便于MyBatis进行扫描。
**config**文件夹目前仅存放application.yml文件用于进行SpringBoot和MyBatis基础配置。
**config**文件夹存放包括
1. ```application.yml```
2. ```application-dev.yml```
3. ```application-test.yml```
4. ```application-prod.yml```
等文件, 用于进行SpringBoot和MyBatis基础配置以及方便进行配置切换。
**nginx**文件夹包括了所有的前端界面和Nginx配置其中
1. nginx.conf用于进行Nginx服务器基础配置关键设置为```include /root/nginx/configs/project.conf```;
2. project.conf.template文件是project.conf的模板文件其可以利用环境变量以及envsubst指令生成project.conf文件详细代码位于```start_nginx.sh```文件)
3. defense文件夹是前端界面文件夹
### Docker相关配置
在docker-ubuntu文件夹中存放的是nginx和springboot的Dockerfile中的```protection_ubuntu```镜像的制作方法,
如果想要重新制作```protection_ubuntu.tar```文件请重新运行docker-ubuntu文件夹中的```Dockerfile```以制作基础ubuntu镜像。可以使用如下指令制作
```shell
docker build -t protection_ubuntu .
```
同时在docker部署之前请执行以下指令以联网获取GRADLE相关资源依赖以便进行断网部署
```shell
export set GRADLE_USER_HOME=./gradle
./gradlew build && ./graldew clean
```
在docker-nginx中存放的是nginx前端服务镜像的制作方法全程理论可以做到断网部署具体制作方法请检查```Dockerfile```以及```start_nginx.sh```
在docker-springboot中存放的是springboot后端服务镜像的制作方法全程理论可以做到断网部署具体制作方法请检查```Dockerfile```以及```start_springboot.sh```
如果想要制作前后端Docker镜像并开启docker容器请使用```load_nginx.sh```和```load_springboot.sh```两个脚本(```load_springboot_multi.sh```仅用于测试Nginx服务器负载均衡用
### 数据库相关配置
目前该项目使用两种数据库Doris和MySQL导入Doris和MySQL表结构可使用如下语句
```shell
# 导入Doris数据库以及表结构IP、端口、用户名均需要修改
mysql -u root -h 192.168.107.89 -P 9030 < realtime_protection/sql/doris-2024-1-23.sql -p
# 导入MySQL数据库以及表结构IP、端口、用户名均需要修改
mysql -u root -h 192.168.107.89 -P 3306 < realtime_protection/sql/mysql-2024-1-23.sql -p
# 导入MySQL数据库以及表结构和表中所有数据IP、端口、用户名均需要修改
mysql -u root -h 192.168.107.89 -P 3306 < realtime_protection/sql/mysql-2024-1-23-with-data.sql -p
```
如果需要导出Doris表结构和MySQL表结构以及数据可使用如下语句
```shell
# 导出MySQL表结构和数据
mysqldump -u root -h 192.168.107.89 -P 3306 --databases realtime_protection > mysql-2024-01-23-with-data.sql -p
# 导出MySQL表结构不导出数据
mysqldump -u root -h 192.168.107.89 -P 3306 --databases realtime_protection -no-data > mysql-2024-01-23-without-data.sql -p
# 导出doris表结构
mysqldump -u root -h 192.168.107.89 -P 9030 --databases command --no-tablespaces -no-data > doris-2024-01-23.sql -p
```

View File

@@ -3,10 +3,10 @@ LABEL authors="Endera"
COPY realtime_protection realtime_protection COPY realtime_protection realtime_protection
ENV SPRINGBOOT_FIRST_URL="localhost:8081" \ ENV SPRINGBOOT_FIRST_URL="172.24.100.53:8081" \
SPRINGBOOT_SECOND_URL="localhost:8082" \ SPRINGBOOT_SECOND_URL="172.24.100.53:8082" \
SPRINGBOOT_THIRD_URL="localhost:8083" \ SPRINGBOOT_THIRD_URL="172.24.100.53:8083" \
SPRINGBOOT_FOURTH_URL="localhost:8084" \ SPRINGBOOT_FOURTH_URL="172.24.100.53:8084" \
NGINX_PORT=8080 \ NGINX_PORT=8080 \
NGINX_SERVER_NAME="localhost" NGINX_SERVER_NAME="localhost"

View File

@@ -7,7 +7,7 @@ export set PROTECTION_NGINX_DIR="$PROTECTION_DIR/src/main/resources/nginx"
export set PROJECT_NGINX_DIR="$BASEDIR/nginx" export set PROJECT_NGINX_DIR="$BASEDIR/nginx"
export set GLOBAL_NGINX_DIR="/etc/nginx" export set GLOBAL_NGINX_DIR="/etc/nginx"
envsubst '${SPRINGBOOT_FIRST_URL} ${SPRINGBOOT_SECOND_URL} ${SPRINGBOOT_THIRD_URL} ${SPRINGBOOT_FOURTH_URL} ${NGINX_PORT} ${NGINX_SERVER_NAME}' < $PROTECTION_NGINX_DIR/configs/project.conf.template > $PROTECTION_NGINX_DIR/configs/project.conf envsubst '${SPRINGBOOT_FIRST_URL} ${SPRINGBOOT_SECOND_URL} ${SPRINGBOOT_THIRD_URL} ${SPRINGBOOT_FOURTH_URL} ${NGINX_PORT} ${NGINX_SERVER_NAME}' < $PROTECTION_NGINX_DIR/configs/project_template.conf > $PROTECTION_NGINX_DIR/configs/project.conf
# 拷贝Nginx配置到指定地址 # 拷贝Nginx配置到指定地址
cp -r $PROTECTION_NGINX_DIR $BASEDIR cp -r $PROTECTION_NGINX_DIR $BASEDIR

View File

@@ -10,10 +10,7 @@ rm -rf .gradle
cd .. cd ..
echo "============start loading springboot============" echo "============start loading springboot============"
docker stop springboot_backend && docker rm springboot_backend docker rm -f $(docker ps -a | grep springboot_backend | awk '{print $1}')
docker stop springboot_backend_1 && docker rm springboot_backend_1
docker stop springboot_backend_2 && docker rm springboot_backend_2
docker stop springboot_backend_3 && docker rm springboot_backend_3
docker image rm protection_springboot protection_ubuntu docker image rm protection_springboot protection_ubuntu
echo "============old containers and images are cleared============" echo "============old containers and images are cleared============"
@@ -26,4 +23,5 @@ echo "============built backend servers============"
docker run -itd --name springboot_backend_1 -p 8081:8081 protection_springboot docker run -itd --name springboot_backend_1 -p 8081:8081 protection_springboot
docker run -itd --name springboot_backend_2 -p 8082:8081 protection_springboot docker run -itd --name springboot_backend_2 -p 8082:8081 protection_springboot
docker run -itd --name springboot_backend_3 -p 8083:8081 protection_springboot docker run -itd --name springboot_backend_3 -p 8083:8081 protection_springboot
docker run -itd --name springboot_backend_4 -p 8084:8081 protection_springboot
echo "============started backend servers============" echo "============started backend servers============"

View File

@@ -3,11 +3,10 @@ package com.realtime.protection.configuration.entity.alert;
import com.fasterxml.jackson.annotation.JsonProperty; import com.fasterxml.jackson.annotation.JsonProperty;
import com.realtime.protection.configuration.entity.task.FiveTupleWithMask; import com.realtime.protection.configuration.entity.task.FiveTupleWithMask;
import com.realtime.protection.configuration.kafka.KafkaMessage;
import lombok.Data; import lombok.Data;
@Data @Data
public class AlertMessage implements KafkaMessage { public class AlertMessage {
@JsonProperty("task_id") @JsonProperty("task_id")
private Long taskId; private Long taskId;

View File

@@ -10,6 +10,7 @@ import java.util.Map;
@Getter @Getter
public enum StateEnum { public enum StateEnum {
// 仅需修改此处即可将任务状态以及对应的State和Num进行对应 // 仅需修改此处即可将任务状态以及对应的State和Num进行对应
// 使用状态模式进行任务状态的管理和切换
PENDING(0, new PendingState()), PENDING(0, new PendingState()),
GENERATING(1, new GeneratingState()), GENERATING(1, new GeneratingState()),
RUNNING(2, new RunningState()), RUNNING(2, new RunningState()),

View File

@@ -15,6 +15,9 @@ import org.springframework.stereotype.Service;
import java.util.List; import java.util.List;
import java.util.Objects; import java.util.Objects;
/**
* 任务状态管理服务类
*/
@Service @Service
@EnableScheduling @EnableScheduling
@Slf4j @Slf4j
@@ -27,6 +30,14 @@ public class StateChangeService {
this.taskService = taskService; this.taskService = taskService;
} }
/**
* 切换任务状态
* @param stateNum 欲切换的任务状态编号编号对应的状态请查看configuration/utils/enums/StateEnum类
* @param taskId 任务ID
* @param inner 此接口是否为内部调用,如果为内部调用,则取消所有的状态检查
* @return 状态切换是否成功
* @throws DorisStartException Doris数据库异常
*/
@DSTransactional @DSTransactional
public Boolean changeState(Integer stateNum, Long taskId, Boolean inner) throws DorisStartException { public Boolean changeState(Integer stateNum, Long taskId, Boolean inner) throws DorisStartException {
Integer originalStateNum = taskService.queryTaskStatus(taskId); Integer originalStateNum = taskService.queryTaskStatus(taskId);
@@ -49,6 +60,7 @@ public class StateChangeService {
return true; return true;
} }
// 切换到states文件夹中查看每一个状态的handle函数
if (!originalState.handle(newState, commandService, taskService, taskId)) { if (!originalState.handle(newState, commandService, taskService, taskId)) {
return false; return false;
} }
@@ -79,6 +91,9 @@ public class StateChangeService {
return !Objects.equals(originalState, StateEnum.GENERATING.getState()); return !Objects.equals(originalState, StateEnum.GENERATING.getState());
} }
/**
* 将任务切换为结束状态
*/
@Scheduled(cron = "0 0/10 * * * ?") @Scheduled(cron = "0 0/10 * * * ?")
@Async @Async
protected void finishTasks() { protected void finishTasks() {

View File

@@ -1,39 +0,0 @@
upstream server_list {
least_conn;
server 192.168.107.89:8081;
}
server {
listen 8080;
server_name localhost;
location / {
root /root/nginx/defense;
index index.html;
try_files $uri $uri/ @router;
}
location @router {
rewrite ^.*$ /index.html last;
}
location /api/ {
add_header 'Access-Control-Allow-Origin' $http_origin;
add_header 'Access-Control-Allow-Credentials' 'true';
add_header 'Access-Control-Allow-Methods' 'GET, POST, DELETE, OPTIONS';
add_header 'Access-Control-Allow-Headers' 'DNT,Authorization,Accept,Origin,Keep-Alive,User-Agent,X-Mx-ReqToken,X-Data-Type,X-Auth-Token,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range';
add_header 'Access-Control-Expose-Headers' 'Content-Length,Content-Range';
if ($request_method = 'OPTIONS') {
add_header 'Access-Control-Max-Age' 1728000;
add_header 'Content-Type' 'text/plain; charset=utf-8';
add_header 'Content-Length' 0;
return 204;
}
proxy_set_header Host $host:$server_port;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_connect_timeout 5;
proxy_pass http://server_list/api/v1/;
}
}

View File

@@ -58,7 +58,7 @@ http {
# include /etc/nginx/conf.d/*.conf; # include /etc/nginx/conf.d/*.conf;
# include /etc/nginx/sites-enabled/*; # include /etc/nginx/sites-enabled/*;
include /root/nginx/configs/*.conf; include /root/nginx/configs/project.conf;
} }