122 Commits

Author SHA1 Message Date
shizhendong
04ddd58d02 feat: ASW-205 新增 application rename 接口 2024-11-28 15:37:07 +08:00
zhangshuai
3611f9226c feat: ASW-199 调整 cron 校验 2024-11-28 15:02:42 +08:00
zhangshuai
0c5c22f567 feat: ASW-199 rcode 补充提交 2024-11-28 14:55:49 +08:00
zhangshuai
e91b0639ce feat: ASW-199 job 接口开发 2024-11-28 14:33:59 +08:00
shizhendong
feedb7a4b4 feat: 补充 workspace_delete 菜单按钮 2024-11-28 14:20:38 +08:00
zhangshuai
28909428fa fix: 修复job 取消后未删除队列中job 2024-11-28 10:30:49 +08:00
zhangshuai
b398e32f2d fix: job 添加job_run和job_edit按钮 2024-11-27 15:09:02 +08:00
zhangshuai
7cafa08aee fix: 修复下载 pcap 文件名称错误 2024-11-26 16:13:47 +08:00
zhangshuai
b189c5d9d0 feat: ASW-192 playbook支持python 脚本类型 2024-11-26 10:13:20 +08:00
zhangshuai
afcbbe30b6 fix:ASW-191 job 增加 artifacts 参数 2024-11-25 15:03:55 +08:00
shizhendong
5444bf2c0b feat: ASW-185 application release 补充编辑接口
1. application release 补充编辑接口
2. sys_menu 添加 application_release 菜单和按钮
2024-11-22 15:22:29 +08:00
shizhendong
3cc928d7a7 feat: ASW-182 新增 tag,release 接口 2024-11-21 14:59:54 +08:00
zhangshuai
70feac12fc fix: 调整 job 执行流程 2024-11-20 16:47:19 +08:00
zhangshuai
0133dc72b2 fix: 调整 job 属性名称 2024-11-20 13:35:30 +08:00
shizhendong
30b3f1c101 refactor: application 模块代码重构,将部分 jgit 代码拆分开 2024-11-20 09:58:05 +08:00
zhangshuai
ec4c58db4e feat: ASW-181 playbook 修改接口开发 2024-11-19 11:34:24 +08:00
shizhendong
9ba7619752 fix: 解决 commit diff 接口重复获取 commitId 信息导致对比信息有误 2024-11-19 10:22:37 +08:00
zhangshuai
62305f77bc fix: playbook 详情缺少 data 属性 2024-11-19 10:11:43 +08:00
zhangshuai
b54b8a77df fix: playbook 添加 edit 按钮 2024-11-19 09:58:46 +08:00
zhangshuai
84c2559ec1 fix: 修复请求 exec playbook 失败释放 session 问题 2024-11-18 17:00:13 +08:00
zhangshuai
fd9f1cb7e4 fix: ASW-176 job 支持 playbook param 参数 2024-11-18 16:46:28 +08:00
zhangshuai
a83da4f1bb fix: ASW-176 job 支持 playbook param 参数 2024-11-18 16:37:40 +08:00
shizhendong
310d393f8a fix: 调整 application 导入导出接口,asw 不记录 condition.attributeType 属性 2024-11-18 10:50:50 +08:00
shizhendong
55a2dabc0d fix: application 新增|导入 接口默认不再添加 icon.png 文件 2024-11-18 09:39:09 +08:00
shizhendong
b368080eaf fix: diff commit 遇到 merge 时,parentId 取 parent[0] 当作 oldCommit 2024-11-15 18:07:01 +08:00
zhangshuai
19369da30b feat: ASW-175 package修改接口开发 2024-11-15 16:31:59 +08:00
shizhendong
d8aed83e7b fix: 字段调整 application signatures ’negate_option‘-> 'negateOption' 2024-11-15 16:26:01 +08:00
shizhendong
6a4aa58d03 fix: 调整 mr merge 提交信息 2024-11-15 16:08:12 +08:00
shizhendong
489e1b9320 fix: 调整 mr merge 提交信息 2024-11-15 15:52:48 +08:00
shizhendong
7f7d0347ce fix: 处理 mr 过程中分支被删除的情况 2024-11-15 15:34:26 +08:00
shizhendong
f322ce25ff fix: application 导入导出接口路径调整 2024-11-15 11:25:56 +08:00
shizhendong
629c9569da feat: package 添加编辑菜单权限 2024-11-15 11:02:51 +08:00
zhangshuai
1ada41d49e fix: ASW-172 修复 asw 资源文件不返回 path 2024-11-15 09:27:15 +08:00
shizhendong
81d58ae868 fix: ASW-167 解决本地仓库 切换main分支 报错问题 2024-11-14 17:22:51 +08:00
zhangshuai
2db15d0f4e fix: ASW-165 调整文件目录,保持规范 2024-11-14 17:02:03 +08:00
zhangshuai
acacbaec72 fix: ASW-164 每三秒检查 playbook 执行状态,保证线程已创建 2024-11-14 14:52:14 +08:00
zhangshuai
b6f6e17d8e Merge remote-tracking branch 'origin/dev-1.1' into dev-1.1 2024-11-14 14:35:08 +08:00
zhangshuai
d14bc9864a fix: ASW-164 修复 job被取消未释放Environment资源 2024-11-14 14:34:49 +08:00
shizhendong
892a961fe1 feat: ASW-149 signature.json 格式更新,支持保存 object 名称类型
1. application 导入导出 数据源切换为 git
2024-11-14 14:05:14 +08:00
shizhendong
fe55a7fcf8 fix: 解决 meta.json 解析错误导致无法查看 application lastCommit 问题 2024-11-13 18:09:25 +08:00
shizhendong
beb8ccdce6 fix: meta.json 格式错误不影响查看 application 2024-11-13 18:02:54 +08:00
shizhendong
facde0a4e1 fix: ASW-162 解决 mr commits,changes 展示信息有误问题 2024-11-13 17:20:24 +08:00
zhangshuai
cba1251bf3 fix: 修复 pcap 按钮名称错误问题 2024-11-13 14:58:53 +08:00
shizhendong
cd61c898e5 fix: merge 动作中,项目克隆到本地后,设置 config.name 2024-11-13 10:37:39 +08:00
shizhendong
37f22f2817 fix: application mr 合并失败后,抛出异常,保持 mr open 状态 2024-11-13 10:14:08 +08:00
zhangshuai
1da79665be fix: ASW-161 pcap接口更新 2024-11-13 09:53:04 +08:00
shizhendong
304895a81b fix: mr 解决冲突接口 改为: clone,ckeckout,merge,add,commit,push 操作流程 2024-11-12 18:11:30 +08:00
zhangshuai
e6c075f653 fix: pcap 补充 jobId 2024-11-12 15:51:52 +08:00
zhangshuai
e2e0934d34 fix: ASW-146 不展示 system 系统用户 2024-11-12 15:20:27 +08:00
zhangshuai
09946cb5f9 fix: ASW-146 不展示 system 系统用户 2024-11-11 10:35:57 +08:00
zhangshuai
d357daa290 fix: ASW-146 修复 job 执行时未修改 environment 状态和未添加session记录 2024-11-11 09:41:41 +08:00
shizhendong
c7c971d508 fix: application diff 接口调整,action 支持 rename,copy 2024-11-08 15:27:59 +08:00
zhangshuai
e5ed7cf47b fix: ASW-148 断开连接后关闭检测线程 2024-11-08 14:58:48 +08:00
zhangshuai
c83e56fc80 fix: ASW-148 使用虚拟线程检查 session 超时 2024-11-08 12:27:38 +08:00
zhangshuai
37299e7180 fix: ASW-148 novnc 实现超时关闭功能 2024-11-08 12:24:09 +08:00
zhangshuai
de19724c1f fix: ASW-148 novnc 实现超时关闭功能 2024-11-08 11:24:51 +08:00
shizhendong
c7d493f150 fix: application 列表页面按照 commit.createdAt 倒叙排序 2024-11-07 16:57:14 +08:00
shizhendong
336c4ed778 fix: application 详情接口查询 app lastCommit 效率优化测试 2024-11-07 15:05:16 +08:00
zhangshuai
e29ffc62d9 Merge remote-tracking branch 'origin/dev-1.1' into dev-1.1 2024-11-07 14:57:00 +08:00
zhangshuai
19090ae43d fix: 修复 获取job 执行结果重复执行问题 2024-11-07 14:56:42 +08:00
shizhendong
2d3040feec fix: application 列表接口查询 app lastCommit 效率优化测试 2024-11-07 14:46:12 +08:00
zhangshuai
bad7573f5d fix: 修复 获取job 执行结果重复执行问题 2024-11-07 09:29:07 +08:00
zhangshuai
912c0f4ae9 fix: ASW-138 playbook 接口调整,增加 type 属性 2024-11-06 17:29:01 +08:00
shizhendong
5ab545a492 fix: ASW-143 application Merge request 状态调整 2024-11-06 16:05:48 +08:00
shizhendong
549126c4ab fix: ASW-134 application 名称从 git 目录中获取 2024-11-05 15:13:03 +08:00
shizhendong
62925cd6b7 fix: 初始化 meta.json 文件格式调整,json 缩进因子改为 2,和前端保持一致 2024-11-05 14:45:51 +08:00
zhangshuai
f2cc7f8cdb fix: 取消 pcap MD5 校验 2024-11-05 14:09:09 +08:00
shizhendong
caf296f49e fix: ASW-132 application basic.json 更名为 meta.json 2024-11-05 11:00:12 +08:00
zhangshuai
a59b97557a fix: 调整 job 与playbook menu 顺序 2024-11-05 10:31:05 +08:00
zhangshuai
2c817a95d6 Merge remote-tracking branch 'origin/dev-1.1' into dev-1.1 2024-11-05 09:55:46 +08:00
zhangshuai
7e7ef56794 fix: job 日志获取前校验 offset 2024-11-05 09:55:18 +08:00
shizhendong
25992507a6 fix: 按文档调整 git update 接口,create,update 动作必须存在内容 2024-11-04 18:04:58 +08:00
shizhendong
e210cb7cd0 fix: 允许新增空文件 2024-11-04 17:39:03 +08:00
shizhendong
bc26bcabd4 fix: 临时处理 asw 和 opensearch 组件角色对应关系 2024-11-04 17:36:18 +08:00
zhangshuai
9ccd0ecc1f feat: ASW-131 job 详情接口开发 2024-11-04 14:57:20 +08:00
zhangshuai
85de514b1a fix: job 执行结束修改结束时间 2024-11-04 14:20:22 +08:00
zhangshuai
8bcbd80ac9 fix:调整 asw 不响应 null 2024-11-04 10:34:37 +08:00
shizhendong
f4a24d3215 feat: commit diff 接口返回 commit详情信息 2024-11-04 10:09:24 +08:00
shizhendong
c05f37af1a fix: branch commits 接口返回分页参数 2024-11-04 09:43:09 +08:00
zhangshuai
5943b5c514 feat:ASW-129 playbook job 定时任务调整 2024-11-04 09:28:20 +08:00
shizhendong
54bcd63e33 feat: ASW-123 新增 application MR diff,conflict,commit 接口
1. 参考 gitlab 实现 merge request 功能
2024-11-01 18:26:05 +08:00
zhangshuai
7decd03eaa fix: ASW-125 以引用 package 不可删除 2024-10-30 16:48:55 +08:00
zhangshuai
69eef064c2 fix: 调整 package 下载文件名称为 filename 2024-10-30 14:29:59 +08:00
zhangshuai
635c4cdade fix: ASW-119 调整 playbook job 执行成功后 pcap name 2024-10-30 13:55:21 +08:00
zhangshuai
5350efe1e0 fix: 调整 android xapk 文件校验 2024-10-30 09:27:49 +08:00
zhangshuai
b1e4a60c46 fix: ASW-122 调整 playbook 下载文件名称 2024-10-29 17:06:40 +08:00
zhangshuai
dee4829605 fix: ASW-122 修复 package name 解析失败问题 2024-10-29 16:58:04 +08:00
zhangshuai
634ac2b8bb fix: 调整 playbook 上传时,保存文件的名称 2024-10-29 13:53:56 +08:00
zhangshuai
2e4f823211 fix: playbook 添加文件校验 2024-10-29 09:37:17 +08:00
shizhendong
300389ebec fix: merge 失败时返回错误状态码 2024-10-28 17:34:24 +08:00
shizhendong
7eb5a6502e fix: merge 失败时返回错误状态码 2024-10-28 17:33:07 +08:00
zhangshuai
2567a1e367 fix: 添加 playbook 按钮 2024-10-28 15:57:45 +08:00
shizhendong
874c95fa9c feat: ASW-113 新增 application MR 接口 2024-10-28 11:01:29 +08:00
zhangshuai
890f5ccda9 fix: 生成 default dashboard 时,先检查 index-pattern 是否存在,存在即删除 2024-10-25 17:09:38 +08:00
zhangshuai
8deb34d134 fix: 内置 workspace 生成 dashboard 2024-10-25 16:51:43 +08:00
shizhendong
3adafde3fe fix: 临时处理 asw 和 opensearch 组件角色对应关系 2024-10-25 16:08:25 +08:00
zhangshuai
b19582293b fix: 调整 workspace 图表模板,之渲染 indexName dashboardName 2024-10-25 14:24:14 +08:00
zhangshuai
0e48531e21 fix: 调整 workspace dashboard name 2024-10-25 11:29:17 +08:00
zhangshuai
e310d0a9a0 Merge remote-tracking branch 'refs/remotes/origin/dev-1.0' into dev-1.1
# Conflicts:
#	src/main/java/net/geedge/asw/module/workspace/service/impl/WorkspaceServiceImpl.java
2024-10-25 11:12:28 +08:00
shizhendong
cb26993d5f fix: 修复 refs/heads/branch 更新错误问题 2024-10-24 17:09:25 +08:00
shizhendong
8a2ece5591 fix: ASW-111 job startTimestamp,endTimeStamp 没有设置值时,返回 null 2024-10-23 15:15:29 +08:00
shizhendong
39a43c3329 Merge remote-tracking branch 'origin/dev-1.0' into dev-1.1
# Conflicts:
#	src/main/java/net/geedge/asw/common/util/Constants.java
2024-10-23 15:05:31 +08:00
shizhendong
806ba7a01b feat: job 列表接口新增 params.q 查询参数
1. 模糊查询 package,environment,playbook
2024-10-23 11:29:51 +08:00
shizhendong
b4b3184bbd fix: 统一列表接口获取分页对象方式 2024-10-23 11:15:56 +08:00
shizhendong
b2c5a5aad4 fix: 调整 job 接口参数校验规则 2024-10-23 10:21:50 +08:00
shizhendong
1a3cf55ff2 fix: list application 接口获取 app lastCommit 使用虚拟线程执行 2024-10-22 17:58:38 +08:00
shizhendong
862b8ccd98 fix: git init 默认提交 README.md 文件,解决新仓库中没有分支问题 2024-10-22 17:04:55 +08:00
shizhendong
a7d9c06388 feat: ASW-105 内置 role 数据
1. Owner,maintainer,developer,guest
2024-10-22 15:45:27 +08:00
zhangshuai
402f37b220 fix: 模糊搜索 appName 不区分大小写 2024-10-22 09:58:58 +08:00
shizhendong
8f451c9375 feat: workspace 创建/删除 同步更新 GIT 仓库
1. workspace 创建时初始化 GIT 仓库
2. workspace 删除时删除 GIT 仓库目录
2024-10-21 16:39:25 +08:00
shizhendong
eb5396437d fix: 调整 workspace 新增/修改接口 visibility 属性校验规则 2024-10-21 16:05:16 +08:00
shizhendong
63fd0ded17 fix: ASW-108 调整 workspace 新增/修改接口 members 属性校验规则 2024-10-21 15:51:56 +08:00
shizhendong
c96e57cb30 Merge remote-tracking branch 'origin/dev-1.0' into dev-1.1
# Conflicts:
#	src/main/resources/db/migration/R__AZ_sys_i18n.sql
2024-10-21 15:42:31 +08:00
shizhendong
d5115bc75c fix: application 列表接口查询 app lastCommit 效率优化测试 2024-10-21 15:35:38 +08:00
shizhendong
4a5783b20e fix: workspace 本地仓库目录修改为:./workspace/{workspace_id} 2024-10-21 14:23:02 +08:00
shizhendong
6c6327382f fix: branch 强制删除,解决未合并的分支无法删除问题 2024-10-18 15:23:09 +08:00
shizhendong
d09ab8483c feat: branch 列表属性添加 'default',表示是否为默认分支 2024-10-18 15:11:56 +08:00
shizhendong
cc5788cf0b fix: 调整 applicaiton 接口校验规则 2024-10-17 18:09:15 +08:00
shizhendong
878d9d5d89 fix: application 返回 icon base64 字符串添加 ’data:image/png;base64,‘ 前缀 2024-10-17 15:01:24 +08:00
shizhendong
08f873e8cd fix: application 列表接口支持名称搜索
1. application 列表接口支持名称搜索
2. application 列表接口返回 icon 字段
3. master 改为 main, 初始分支是 main 分支
2024-10-17 14:48:55 +08:00
shizhendong
dcc02903f7 feat: new application 时,basic.josn signature.json 初始化文件内容
1. 默认文件内容定义在 application.yml 中
2024-10-16 17:43:25 +08:00
shizhendong
938925e76c feat: ASW-88 本地 GIT 仓库管理 application 文件
1. 使用 JGIT 完成对 application 文件的操作
2024-10-16 16:13:56 +08:00
108 changed files with 6336 additions and 2217 deletions

15
pom.xml
View File

@@ -186,11 +186,26 @@
<version>7.0.0.202409031743-r</version> <version>7.0.0.202409031743-r</version>
</dependency> </dependency>
<!-- https://mvnrepository.com/artifact/org.eclipse.jgit/org.eclipse.jgit.archive -->
<dependency>
<groupId>org.eclipse.jgit</groupId>
<artifactId>org.eclipse.jgit.archive</artifactId>
<version>7.0.0.202409031743-r</version>
</dependency>
<dependency> <dependency>
<groupId>org.springframework.boot</groupId> <groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-freemarker</artifactId> <artifactId>spring-boot-starter-freemarker</artifactId>
</dependency> </dependency>
<!-- https://mvnrepository.com/artifact/net.lingala.zip4j/zip4j -->
<dependency>
<groupId>net.lingala.zip4j</groupId>
<artifactId>zip4j</artifactId>
<version>2.11.5</version>
</dependency>
</dependencies> </dependencies>
<build> <build>

View File

@@ -0,0 +1,109 @@
package net.geedge.asw.common.config;
import cn.hutool.core.io.FileUtil;
import cn.hutool.http.HttpRequest;
import cn.hutool.http.HttpResponse;
import cn.hutool.http.Method;
import cn.hutool.json.JSONObject;
import cn.hutool.log.Log;
import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
import com.baomidou.mybatisplus.core.conditions.update.LambdaUpdateWrapper;
import net.geedge.asw.common.util.Constants;
import net.geedge.asw.common.util.T;
import net.geedge.asw.module.environment.entity.EnvironmentEntity;
import net.geedge.asw.module.environment.entity.EnvironmentSessionEntity;
import net.geedge.asw.module.environment.service.IEnvironmentService;
import net.geedge.asw.module.environment.service.IEnvironmentSessionService;
import net.geedge.asw.module.job.entity.JobEntity;
import net.geedge.asw.module.job.util.JobQueueManager;
import net.geedge.asw.module.job.service.IJobService;
import net.geedge.asw.module.job.util.JobConstant;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.CommandLineRunner;
import org.springframework.stereotype.Component;
import java.util.List;
/**
* setup初始化操作
*/
@Component
public class SetupRunner implements CommandLineRunner{
private static final Log log = Log.get();
@Autowired
private IJobService jobService;
@Autowired
private JobQueueManager jobQueueManager;
@Autowired
private IEnvironmentService environmentService;
@Autowired
private IEnvironmentSessionService environmentSessionService;
@Override
public void run(String... args) throws Exception {
log.info("Setup inited");
List<JobEntity> pendingJobs = jobService.list(new LambdaQueryWrapper<JobEntity>().eq(JobEntity::getStatus, JobConstant.JobStatus.PENDING.getValue()));
pendingJobs.forEach(jobQueueManager::addJob);
log.info("[SetupRunner] [init pending job to JobQueueManager]");
log.info("[SetupRunner] [begin interrupted running job]");
List<JobEntity> runningJobs = jobService.list(new LambdaQueryWrapper<JobEntity>().eq(JobEntity::getStatus, JobConstant.JobStatus.RUNNING.getValue()));
for (JobEntity runningJob : runningJobs) {
String id = runningJob.getId();
EnvironmentEntity environment = environmentService.getById(runningJob.getEnvId());
JSONObject paramJSONObject = environment.getParamJSONObject();
String url = paramJSONObject.getStr("url");
String token = paramJSONObject.getStr("token");
HttpRequest requestStatus = T.HttpUtil.createGet(String.format("%s/api/v1/env/playbook/%s", url, runningJob.getId()));
requestStatus.header("Authorization", token);
HttpResponse response = requestStatus.execute();
if (response.isOk()){
String body = response.body();
JSONObject result = T.JSONUtil.toBean(body, JSONObject.class);
JSONObject data = result.getJSONObject("data");
String status = data.getStr("status");
if (JobConstant.JobStatus.RUNNING.getValue().equals(status)){
HttpRequest request = T.HttpUtil.createRequest(Method.DELETE, String.format("%s/api/v1/env/playbook/%s", url, runningJob.getId()));
request.header("Authorization", token);
request.execute();
}
}
Thread runningThread = Constants.RUNNING_JOB_THREAD.get(id);
if (runningThread != null) {
runningThread.interrupt();
}
Thread resultThread = Constants.RESULT_JOB_THREAD.get(id);
if (resultThread != null) {
resultThread.interrupt();
}
EnvironmentSessionEntity session = environmentSessionService.getOne(new LambdaQueryWrapper<EnvironmentSessionEntity>()
.eq(EnvironmentSessionEntity::getJobId, id)
.eq(EnvironmentSessionEntity::getStatus, 1));
if (T.ObjectUtil.isNotEmpty(session)) {
environmentService.removeSession(session.getId());
}
T.FileUtil.appendString("Job execution interrupted.", FileUtil.file(runningJob.getLogPath()), "UTF-8");
// update state
jobService.update(new LambdaUpdateWrapper<JobEntity>()
.eq(JobEntity::getId, id)
.set(JobEntity::getStatus, "failed")
);
}
log.info("[SetupRunner] [interrupted running job end!]");
}
}

View File

@@ -0,0 +1,17 @@
package net.geedge.asw.common.config;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
@Configuration
public class ThreadPoolConfig {
@Bean
public ExecutorService virtualThreadExecutor() {
return Executors.newVirtualThreadPerTaskExecutor();
}
}

View File

@@ -4,8 +4,8 @@ import cn.hutool.log.Log;
import jakarta.annotation.PostConstruct; import jakarta.annotation.PostConstruct;
import net.geedge.asw.common.util.T; import net.geedge.asw.common.util.T;
import net.geedge.asw.module.environment.job.JobEnvironmentStatusChecker; import net.geedge.asw.module.environment.job.JobEnvironmentStatusChecker;
import net.geedge.asw.module.runner.job.JobPlaybookExecResultChecker; import net.geedge.asw.module.job.job.JobPlaybookExecResultChecker;
import net.geedge.asw.module.runner.job.JobPlaybookExecutor; import net.geedge.asw.module.job.job.JobPlaybookExecutor;
import net.geedge.asw.module.sys.service.ISysConfigService; import net.geedge.asw.module.sys.service.ISysConfigService;
import org.quartz.*; import org.quartz.*;
import org.springframework.beans.factory.annotation.Autowired; import org.springframework.beans.factory.annotation.Autowired;
@@ -70,8 +70,10 @@ public class JobConfig {
public void init() throws SchedulerException { public void init() throws SchedulerException {
// JobEnvironmentStatusChecker // JobEnvironmentStatusChecker
createCronScheduleJob(JobEnvironmentStatusChecker(), environment.getProperty("asw.cron.JobEnvironmentStatusChecker", "0 0/1 * * * ? *")); createCronScheduleJob(JobEnvironmentStatusChecker(), environment.getProperty("asw.cron.JobEnvironmentStatusChecker", "0 0/1 * * * ? *"));
createCronScheduleJob(JobPlaybookExecutor(), environment.getProperty("asw.cron.JobPlaybookExecutor", "0 0/1 * * * ? *")); // JobPlaybookExecutor
createCronScheduleJob(JobPlaybookExecResultChecker(), environment.getProperty("asw.cron.JobPlaybookExecResultChecker", "0/30 * * * * ?")); createCronScheduleJob(JobPlaybookExecutor(), environment.getProperty("asw.cron.JobPlaybookExecutor", "0/30 * * * * ?"));
// JobPlaybookExecResultChecker
createCronScheduleJob(JobPlaybookExecResultChecker(), environment.getProperty("asw.cron.JobPlaybookExecResultChecker", "0/10 * * * * ?"));
} }
/** /**

View File

@@ -6,6 +6,7 @@ import cn.hutool.core.util.ObjectUtil;
import cn.hutool.json.JSONObject; import cn.hutool.json.JSONObject;
import cn.hutool.log.Log; import cn.hutool.log.Log;
import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper; import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
import com.baomidou.mybatisplus.core.conditions.update.LambdaUpdateWrapper;
import net.geedge.asw.common.util.Constants; import net.geedge.asw.common.util.Constants;
import net.geedge.asw.common.util.T; import net.geedge.asw.common.util.T;
import net.geedge.asw.module.environment.entity.EnvironmentEntity; import net.geedge.asw.module.environment.entity.EnvironmentEntity;
@@ -31,7 +32,6 @@ public class EnvironmentNovncWebSocketHandler extends TextWebSocketHandler {
* env id * env id
*/ */
private String envId; private String envId;
/** /**
* session * session
*/ */
@@ -42,14 +42,20 @@ public class EnvironmentNovncWebSocketHandler extends TextWebSocketHandler {
*/ */
private String userId; private String userId;
private long lastReceivedTime;
private Integer sessionTimeout;
private IEnvironmentService environmentService; private IEnvironmentService environmentService;
private IEnvironmentSessionService environmentSessionService; private IEnvironmentSessionService environmentSessionService;
public EnvironmentNovncWebSocketHandler(IEnvironmentService environmentService, IEnvironmentSessionService environmentSessionService) { public EnvironmentNovncWebSocketHandler() {
this.environmentService = environmentService; }
public EnvironmentNovncWebSocketHandler(IEnvironmentService deviceService, IEnvironmentSessionService environmentSessionService, Integer sessionTimeout) {
this.environmentService = deviceService;
this.environmentSessionService = environmentSessionService; this.environmentSessionService = environmentSessionService;
this.sessionTimeout = sessionTimeout;
} }
private void initFieldVal(WebSocketSession session) { private void initFieldVal(WebSocketSession session) {
@@ -57,8 +63,41 @@ public class EnvironmentNovncWebSocketHandler extends TextWebSocketHandler {
this.sessionId = (String) session.getAttributes().get("sessionId"); this.sessionId = (String) session.getAttributes().get("sessionId");
this.userId = (String) session.getAttributes().get("userId"); this.userId = (String) session.getAttributes().get("userId");
Constants.ENV_NOVNC_WEBSOCKET_SESSION.put(sessionId, session); Constants.ENV_NOVNC_WEBSOCKET_SESSION.put(sessionId, session);
lastReceivedTime = System.currentTimeMillis(); // 初始化接收时间
Thread checkSessionTimeout = startConnectionMonitor(session);
session.getAttributes().put("checkSessionTimeoutThread", checkSessionTimeout);
} }
private Thread startConnectionMonitor(WebSocketSession session) {
Thread thread = Thread.ofVirtual().start(() -> {
while (true) {
try {
if (System.currentTimeMillis() - lastReceivedTime > sessionTimeout) {
if (session.isOpen()) {
log.info("current no connection info, clean no real use connection finshed, sessionId: {}", sessionId);
// update session status
environmentSessionService.update(new LambdaUpdateWrapper<EnvironmentSessionEntity>()
.set(EnvironmentSessionEntity::getStatus, 2)
.eq(EnvironmentSessionEntity::getId, sessionId));
session.close(CloseStatus.NORMAL.withReason("current no connection info, clean no real use connection finshed."));
break;
}
}
Thread.sleep(300000);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
break;
} catch (IOException e) {
log.error(e, "Error clean no real use connection");
}
}
});
return thread;
}
@Override @Override
public void afterConnectionEstablished(WebSocketSession session) throws Exception { public void afterConnectionEstablished(WebSocketSession session) throws Exception {
super.afterConnectionEstablished(session); super.afterConnectionEstablished(session);
@@ -139,6 +178,7 @@ public class EnvironmentNovncWebSocketHandler extends TextWebSocketHandler {
WebSocket envSocket = (WebSocket) session.getAttributes().get("envWebsocket"); WebSocket envSocket = (WebSocket) session.getAttributes().get("envWebsocket");
if (envSocket != null) { if (envSocket != null) {
envSocket.sendBinary(message.getPayload(), true); envSocket.sendBinary(message.getPayload(), true);
lastReceivedTime = System.currentTimeMillis(); // 更新接收时间
} }
} catch (Exception e) { } catch (Exception e) {
log.error(e, "[handleBinaryMessage] [error]"); log.error(e, "[handleBinaryMessage] [error]");
@@ -152,6 +192,11 @@ public class EnvironmentNovncWebSocketHandler extends TextWebSocketHandler {
if (envWebsocket != null) { if (envWebsocket != null) {
envWebsocket.sendClose(WebSocket.NORMAL_CLOSURE, "Normal closure"); envWebsocket.sendClose(WebSocket.NORMAL_CLOSURE, "Normal closure");
} }
Thread checkSessionTimeoutThread = (Thread) session.getAttributes().get("checkSessionTimeoutThread");
if (checkSessionTimeoutThread != null) {
checkSessionTimeoutThread.interrupt();
}
Constants.ENV_NOVNC_WEBSOCKET_SESSION.remove(sessionId); Constants.ENV_NOVNC_WEBSOCKET_SESSION.remove(sessionId);
super.afterConnectionClosed(session, status); super.afterConnectionClosed(session, status);
} }

View File

@@ -3,6 +3,7 @@ package net.geedge.asw.common.config.websocket;
import net.geedge.asw.module.environment.service.IEnvironmentService; import net.geedge.asw.module.environment.service.IEnvironmentService;
import net.geedge.asw.module.environment.service.IEnvironmentSessionService; import net.geedge.asw.module.environment.service.IEnvironmentSessionService;
import org.springframework.beans.factory.annotation.Autowired; import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Configuration; import org.springframework.context.annotation.Configuration;
import org.springframework.web.socket.config.annotation.EnableWebSocket; import org.springframework.web.socket.config.annotation.EnableWebSocket;
import org.springframework.web.socket.config.annotation.WebSocketConfigurer; import org.springframework.web.socket.config.annotation.WebSocketConfigurer;
@@ -12,6 +13,10 @@ import org.springframework.web.socket.config.annotation.WebSocketHandlerRegistry
@EnableWebSocket @EnableWebSocket
public class WebSocketConfig implements WebSocketConfigurer { public class WebSocketConfig implements WebSocketConfigurer {
@Value("${session.timeout:1800000}") // 默认为 30 分钟
private Integer sessionTimeout;
@Autowired @Autowired
private IEnvironmentService deviceService; private IEnvironmentService deviceService;
@@ -20,7 +25,7 @@ public class WebSocketConfig implements WebSocketConfigurer {
@Override @Override
public void registerWebSocketHandlers(WebSocketHandlerRegistry registry) { public void registerWebSocketHandlers(WebSocketHandlerRegistry registry) {
registry.addHandler(new EnvironmentNovncWebSocketHandler(deviceService, environmentSessionService), "/api/v1/env/{envId}/session/{sessionId}/novnc") registry.addHandler(new EnvironmentNovncWebSocketHandler(deviceService, environmentSessionService, sessionTimeout), "/api/v1/env/{envId}/session/{sessionId}/novnc")
.addInterceptors(new EnvironmentWebSocketInterceptor()) .addInterceptors(new EnvironmentWebSocketInterceptor())
.setAllowedOrigins("*"); .setAllowedOrigins("*");

View File

@@ -6,6 +6,7 @@ import java.io.File;
import java.util.HashMap; import java.util.HashMap;
import java.util.List; import java.util.List;
import java.util.Map; import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
public class Constants { public class Constants {
@@ -19,10 +20,6 @@ public class Constants {
*/ */
public static final String TEMP_PATH = System.getProperty("user.dir") + File.separator + "tmp"; public static final String TEMP_PATH = System.getProperty("user.dir") + File.separator + "tmp";
/**
* playbook dir
*/
public static File PLAYBOOK_FILES_DIR = T.FileUtil.file(T.WebPathUtil.getRootPath(), "playbook_files");
/** /**
* 国际化语言列表 * 国际化语言列表
@@ -104,15 +101,73 @@ public class Constants {
public static final Map<String, WebSocketSession> ENV_TERMINAL_WEBSOCKET_SESSION = T.MapUtil.newHashMap(); public static final Map<String, WebSocketSession> ENV_TERMINAL_WEBSOCKET_SESSION = T.MapUtil.newHashMap();
public static final ConcurrentHashMap<String, Thread> RUNNING_JOB_THREAD = new ConcurrentHashMap<>();
public static final ConcurrentHashMap<String, Thread> RESULT_JOB_THREAD = new ConcurrentHashMap<>();
/** /**
* Android package type * Android package type
*/ */
public static final List<String> ANDROID_PACKAGE_TYPE_LIST = T.ListUtil.of("xapk", "apk"); public static final List<String> ANDROID_PACKAGE_TYPE_LIST = T.ListUtil.of("xapk", "apk");
/**
* job cfg type
*/
public static final List<String> JOB_CFG_TYPE_LIST= T.ListUtil.of("asap", "cron");
/**
* job cfg status
*/
public static final List<String> JOB_CFG_STATUS_LIST= T.ListUtil.of("enabled", "disabled");
public static final String EMPTY_FILE_MD5 = "d41d8cd98f00b204e9800998ecf8427e"; public static final String EMPTY_FILE_MD5 = "d41d8cd98f00b204e9800998ecf8427e";
/** /**
* tid -> jobId 用于获取 job 运行结果 * 系统内置角色
*/ */
public static final Map<String, String> PLAYBOOK_EXECUTOR_RESULT = T.MapUtil.newHashMap(); public static enum BuiltInRoleEnum {
OWNER("owner"),
MAINTAINER("maintainer"),
DEVELOPER("developer"),
GUEST("guest");
private String id;
BuiltInRoleEnum(String id) {
this.id = id;
}
public String getId() {
return id;
}
}
/**
* 文件类型
*/
public static enum FileTypeEnum {
PCAP("pcap"),
PACKAGE("package"),
PLAYBOOK("playbook"),
JOB("job"),
RELEASE("release");
private String type;
FileTypeEnum(String type) {
this.type = type;
}
public String getType() {
return type;
}
}
} }

View File

@@ -0,0 +1,14 @@
package net.geedge.asw.common.util;
import java.io.File;
public class FileResourceUtil {
private static String ROOT_PATH = T.WebPathUtil.getRootPath();
public static File createFile(String resourcePath, String workspaceId, String type, String id, String name) {
String sub = T.StrUtil.sub(id, 0, 2);
File file = T.FileUtil.file(ROOT_PATH, resourcePath, workspaceId, type, sub, id, name);
return file;
}
}

View File

@@ -29,6 +29,7 @@ public enum RCode {
SYS_ROLE_BUILT_IN(100017, "Built-in role are not allowed to delete or update"), SYS_ROLE_BUILT_IN(100017, "Built-in role are not allowed to delete or update"),
SYS_ROLE_NOT_DELETE(100018, "Used role cannot be deleted"), SYS_ROLE_NOT_DELETE(100018, "Used role cannot be deleted"),
SYS_USER_OLDPWD_INCORRECT(100019, "Incorrect old password. Please try again."), SYS_USER_OLDPWD_INCORRECT(100019, "Incorrect old password. Please try again."),
SYS_SYSTEM_USER_NOT_LOGIN(100020, "System user cannot login."),
// Application // Application
@@ -56,6 +57,19 @@ public enum RCode {
PACKAGE_ID_CANNOT_EMPTY(202001, "package id cannot be empty"), PACKAGE_ID_CANNOT_EMPTY(202001, "package id cannot be empty"),
PACKAGE_DESCRIPTION_CANNOT_EMPTY(202002, "package description cannot be empty"), PACKAGE_DESCRIPTION_CANNOT_EMPTY(202002, "package description cannot be empty"),
PACKAGE_FILE_TYPE_ERROR(202003, "package invalid file"), PACKAGE_FILE_TYPE_ERROR(202003, "package invalid file"),
PACKAGE_CANNOT_DELETE(202004, "The referenced package cannot be deleted"),
// GIT
GIT_COMMIT_CONFLICT_ERROR(203001, "Commit failed; fix conflicts and then commit the result"),
GIT_MERGE_FAILED(203002, "Merge failed,error message: {0}"),
GIT_PARENT_COMMITID_NOT_FOUND(203003, "Parent commitId not found"),
GIT_BINARY_CONFLICT_ERROR(203004, "Binary file conflict found; resolve conflicts in binary files manually"),
GIT_MERGE_NOT_SUPPORTED(203005, "Cannot merge in the {0} state"),
GIT_MERGE_TARGET_BRANCH_NOT_EXIST(203006, "The target branch {0} does not exist."),
GIT_TAG_ALREADY_EXISTS(203007, "Tag {0} already exists"),
GIT_TAG_NOT_FOUND(203008, "Tag {0} not found"),
GIT_TAG_ALREADY_IN_USE(203009,"Tag is already in use. Choose another tag."),
// Runner // Runner
@@ -65,6 +79,7 @@ public enum RCode {
// Playbook // Playbook
PLAYBOOK_ID_CANNOT_EMPTY(302001, "playbook id cannot be empty"), PLAYBOOK_ID_CANNOT_EMPTY(302001, "playbook id cannot be empty"),
PLAYBOOK_NAME_DUPLICATE(302002, "playbook name duplicate "), PLAYBOOK_NAME_DUPLICATE(302002, "playbook name duplicate "),
PLAYBOOK_INVALID_FILE(302003, "playbook Invalid file"),
// Workspace // Workspace
WORKSPACE_ID_CANNOT_EMPTY(401001, "workspace id cannot be empty"), WORKSPACE_ID_CANNOT_EMPTY(401001, "workspace id cannot be empty"),
@@ -91,6 +106,18 @@ public enum RCode {
ENVIRONMENT_ID_CANNOT_EMPTY(601005, "environment id cannot be empty"), ENVIRONMENT_ID_CANNOT_EMPTY(601005, "environment id cannot be empty"),
//job
JOB_CFG_TYPE_CANNOT_EMPTY(701001, "Job configuration type cannot be empty"),
JOB_CFG_STATUS_CANNOT_EMPTY(701002, "Job configuration status cannot be empty"),
JOB_CFG_STATUS_ERROR(701003, "Job configuration status error"),
JOB_CFG_TYPE_ERROR(701004, "Job configuration type error"),
JOB_CFG_CRON_CANNOT_EMPTY(701005, "Job configuration cron cannot be empty"),
JOB_CFG_CRON_ERROR(701006, "Job configuration cron is not a valid cron expression"),
JOB_CFG_NANE_ALREADY_EXISTS(701007, "Job configuration name already exists"),
JOB_CFG_NOT_EXIST(701008, "Job configuration does not exist"),
SUCCESS(200, "success"); // 成功 SUCCESS(200, "success"); // 成功

View File

@@ -310,27 +310,6 @@ public class T {
* @author xiaoleilu * @author xiaoleilu
*/ */
public static class PageUtil extends cn.hutool.core.util.PageUtil { public static class PageUtil extends cn.hutool.core.util.PageUtil {
public static final Integer DEFAULT_PAGENO = 1;
public static final Integer DEFAULT_PAGESIZE = 20;
public static Page getPage(Map<String, Object> params) {
// 分页参数
Integer pageNo = T.MapUtil.getInt(params, "current", DEFAULT_PAGENO);
Integer pageSize = T.MapUtil.getInt(params, "size", DEFAULT_PAGESIZE);
if (pageSize == -1) {
pageNo = 0;
pageSize = Integer.MAX_VALUE;
}
Page page = Page.of(pageNo, pageSize);
String orderBy = T.MapUtil.getStr(params, "orderBy");
if (T.StrUtil.isNotEmpty(orderBy)) {
page.addOrder(T.PageUtil.decodeOrderByStr(orderBy));
}
return page;
}
public static OrderItem decodeOrderByStr(String orderBy) { public static OrderItem decodeOrderByStr(String orderBy) {
if (cn.hutool.core.util.StrUtil.isBlank(orderBy)) { if (cn.hutool.core.util.StrUtil.isBlank(orderBy)) {
return null; return null;

View File

@@ -1,277 +1,140 @@
package net.geedge.asw.module.app.controller; package net.geedge.asw.module.app.controller;
import cn.hutool.json.JSONObject; import cn.hutool.json.JSONObject;
import cn.hutool.log.Log;
import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import jakarta.servlet.http.HttpServletResponse; import jakarta.servlet.http.HttpServletResponse;
import net.geedge.asw.common.util.ASWException; import net.geedge.asw.common.util.ASWException;
import net.geedge.asw.common.util.R; import net.geedge.asw.common.util.R;
import net.geedge.asw.common.util.RCode; import net.geedge.asw.common.util.RCode;
import net.geedge.asw.common.util.T; import net.geedge.asw.common.util.T;
import net.geedge.asw.module.app.entity.*; import net.geedge.asw.module.app.entity.ApplicationEntity;
import net.geedge.asw.module.app.service.*; import net.geedge.asw.module.app.service.IApplicationService;
import net.geedge.asw.module.workspace.entity.WorkspaceEntity; import net.geedge.asw.module.workspace.entity.WorkspaceEntity;
import net.geedge.asw.module.workspace.service.IWorkspaceService; import net.geedge.asw.module.workspace.service.IWorkspaceService;
import org.springframework.beans.factory.annotation.Autowired; import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.transaction.annotation.Transactional;
import org.springframework.web.bind.annotation.*; import org.springframework.web.bind.annotation.*;
import org.springframework.web.multipart.MultipartFile; import org.springframework.web.multipart.MultipartFile;
import java.io.IOException; import java.io.IOException;
import java.util.ArrayList; import java.nio.charset.StandardCharsets;
import java.util.Collection;
import java.util.List; import java.util.List;
import java.util.Map; import java.util.Map;
import java.util.stream.Collectors; import java.util.stream.Collectors;
@RestController @RestController
@RequestMapping("/api/v1/application") @RequestMapping("/api/v1/workspace")
public class ApplicationController { public class ApplicationController {
private static final Log log = Log.get();
@Autowired @Autowired
private IWorkspaceService workspaceService; private IWorkspaceService workspaceService;
@Autowired @Autowired
private IApplicationService applicationService; private IApplicationService applicationService;
@Autowired @GetMapping("/{workspaceId}/branch/{branchName}/application/{applicationName}")
private IApplicationSignatureService signatureService; public R infoApplication(@PathVariable("workspaceId") String workspaceId,
@PathVariable("branchName") String branchName,
@Autowired @PathVariable("applicationName") String applicationName) {
private IApplicationNoteService noteService; Map<Object, Object> record = applicationService.infoApplication(workspaceId, branchName, applicationName);
return R.ok().putData("record", record);
@Autowired
private IApplicationHrefService hrefService;
@Autowired
private IApplicationAttachmentService attachmentService;
@GetMapping("/{id}")
public R detail(@PathVariable("id") String id, String workspaceId) {
T.VerifyUtil.is(workspaceId).notNull();
ApplicationEntity entity = applicationService.detail(id, workspaceId);
if (T.ObjectUtil.isNull(entity)) {
throw new ASWException(RCode.APP_NOT_EXIST);
}
return R.ok().putData("record", entity);
} }
@GetMapping("/{workspaceId}/branch/{branchName}/application")
@GetMapping public R listApplication(@PathVariable("workspaceId") String workspaceId,
public R list(@RequestParam Map<String, Object> params) { @PathVariable("branchName") String branchName,
T.VerifyUtil.is(params).notNull() @RequestParam(value = "q", required = false) String q) {
.and(T.MapUtil.getStr(params, "workspaceId")).notEmpty(RCode.WORKSPACE_ID_CANNOT_EMPTY); List<Map<Object, Object>> records = applicationService.listApplication(workspaceId, branchName, q);
Page page = applicationService.queryList(params);
return R.ok(page);
}
@PostMapping
@Transactional(rollbackFor = Exception.class)
public R add(@RequestParam(required = true) String basic,
@RequestParam(required = false) String signature,
@RequestParam(required = false) String note,
@RequestParam(required = false) String hrefs,
@RequestParam(required = false, value = "files") List<MultipartFile> fileList) {
// validate
ApplicationEntity entity;
try {
entity = T.JSONUtil.toBean(basic, ApplicationEntity.class);
if (T.StrUtil.isNotEmpty(signature)) {
ApplicationSignatureEntity signatureEntity = T.JSONUtil.toBean(signature, ApplicationSignatureEntity.class);
entity.setSignature(signatureEntity);
}
if (T.StrUtil.isNotEmpty(note)) {
ApplicationNoteEntity noteEntity = T.JSONUtil.toBean(note, ApplicationNoteEntity.class);
entity.setNote(noteEntity);
}
if (T.StrUtil.isNotEmpty(hrefs)) {
T.JSONUtil.toList(hrefs, ApplicationHrefEntity.class);
}
} catch (Exception e) {
log.error(e, "[add] [param format error]");
throw new ASWException(RCode.ERROR);
}
T.VerifyUtil.is(entity).notNull()
.and(entity.getName()).notEmpty(RCode.APP_NAME_CANNOT_EMPTY)
.and(entity.getWorkspaceId()).notEmpty(RCode.WORKSPACE_ID_CANNOT_EMPTY);
// save application
ApplicationEntity applicationEntity = applicationService.saveApplication(entity);
// save attachment
fileList = T.CollUtil.defaultIfEmpty(fileList, new ArrayList<>());
for (MultipartFile file : fileList) {
attachmentService.saveAttachment(file.getResource(), applicationEntity.getId());
}
// save href
if (T.StrUtil.isNotEmpty(hrefs)) {
List<ApplicationHrefEntity> hrefList = T.JSONUtil.toList(hrefs, ApplicationHrefEntity.class);
hrefService.updateBatchHref(applicationEntity.getId(), hrefList);
}
return R.ok().putData("id", applicationEntity.getId());
}
@PutMapping
public R update(@RequestBody ApplicationEntity entity) {
T.VerifyUtil.is(entity).notNull()
.and(entity.getId()).notEmpty(RCode.ID_CANNOT_EMPTY)
.and(entity.getName()).notEmpty(RCode.NAME_CANNOT_EMPTY)
//.and(entity.getSignature()).notEmpty(RCode.APP_SURROGATES_CANNOT_EMPTY)
//.and(entity.getNote()).notEmpty(RCode.APP_PROPERTIES_CANNOT_EMPTY)
.and(entity.getWorkspaceId()).notEmpty(RCode.WORKSPACE_ID_CANNOT_EMPTY);
ApplicationEntity applicationEntity = applicationService.updateApplication(entity);
return R.ok().putData("id", applicationEntity.getId());
}
@PutMapping("/{id}/basic")
public R basic(@PathVariable String id, @RequestBody ApplicationEntity entity) {
T.VerifyUtil.is(entity).notNull()
.and(entity.getName()).notEmpty(RCode.NAME_CANNOT_EMPTY)
.and(entity.getWorkspaceId()).notEmpty(RCode.WORKSPACE_ID_CANNOT_EMPTY);
entity.setId(id);
ApplicationEntity app = applicationService.updateBasic(entity);
return R.ok().putData("id", app.getId());
}
@PutMapping("/{applicationId}/signature")
public R updateSignature(@PathVariable("applicationId") String applicationId, @RequestBody ApplicationSignatureEntity signature) {
T.VerifyUtil.is(signature).notNull()
.and(signature.getContent()).notEmpty(RCode.APP_SURROGATES_CANNOT_EMPTY)
.and(signature.getContent()).json(RCode.APP_SURROGATES_CANNOT_EMPTY);
signatureService.saveSignature(signature, applicationId);
return R.ok().putData("id", signature.getId());
}
@PutMapping("/{applicationId}/note")
public R updateNote(@PathVariable("applicationId") String applicationId, @RequestBody ApplicationNoteEntity note) {
T.VerifyUtil.is(note).notNull();
//.and(note.getContent()).notEmpty(RCode.APP_NOTE_CONTENT_CANNOT_EMPTY);
noteService.saveNote(note, applicationId);
return R.ok().putData("id", note.getId());
}
@DeleteMapping
public R delete(String[] ids) {
T.VerifyUtil.is(ids).notEmpty();
applicationService.removeApplication(T.ListUtil.of(ids));
return R.ok();
}
@GetMapping("/{applicationId}/attachment")
public R queryAttachment(@PathVariable String applicationId) {
T.VerifyUtil.is(applicationId).notNull();
List<ApplicationAttachmentEntity> list = attachmentService.list(new LambdaQueryWrapper<ApplicationAttachmentEntity>().eq(ApplicationAttachmentEntity::getApplicationId, applicationId));
return R.ok().putData("records", list);
}
@GetMapping("/{applicationId}/attachment/{attachmentId}")
public void downloadAttachment(HttpServletResponse response, @PathVariable String applicationId, @PathVariable String attachmentId) throws IOException {
T.VerifyUtil.is(applicationId).notNull()
.and(attachmentId).notNull();
attachmentService.download(response, applicationId, attachmentId);
}
@PostMapping("/{applicationId}/attachment")
public R uploadAttachment(@PathVariable String applicationId, @RequestParam("files") List<MultipartFile> fileList) {
List<ApplicationAttachmentEntity> recordList = T.ListUtil.list(true);
for (int i = 0; i < fileList.size(); i++) {
MultipartFile file = fileList.get(i);
ApplicationAttachmentEntity attachmentEntity = attachmentService.saveAttachment(file.getResource(), applicationId);
recordList.add(attachmentEntity);
}
return R.ok().putData("records", recordList);
}
@DeleteMapping("/{applicationId}/attachment")
public R removedAttachment(@PathVariable String applicationId, @RequestParam String ids) {
attachmentService.removedAttachment(applicationId, ids);
return R.ok();
}
@GetMapping("/{applicationId}/signature")
public R querySignature(@PathVariable String applicationId) {
T.VerifyUtil.is(applicationId).notNull();
List<ApplicationSignatureEntity> signatureList = signatureService.queryList(applicationId);
return R.ok().putData("records", signatureList);
}
@GetMapping("/{applicationId}/signature/{oldVersion}/{newVersion}")
public R signatureCompare(@PathVariable("applicationId") String applicationId,
@PathVariable("oldVersion") String oldVersion,
@PathVariable("newVersion") String newVersion) {
List<ApplicationSignatureEntity> list = signatureService.compare(applicationId, oldVersion, newVersion);
return R.ok().putData("records", list);
}
@PutMapping("/{applicationId}/signature/{version}/restore")
public R restore(@PathVariable("applicationId") String applicationId,
@PathVariable("version") String version) {
signatureService.restore(applicationId, version);
return R.ok();
}
// application href
@GetMapping("/{applicationId}/href")
public R queryHref(@PathVariable String applicationId) {
List<ApplicationHrefEntity> entityList = hrefService.queryList(applicationId);
return R.ok().putData("records", entityList);
}
@RequestMapping(value = "/{applicationId}/href", method = {RequestMethod.POST, RequestMethod.PUT})
public R updateBatchHref(@PathVariable String applicationId, @RequestBody List<ApplicationHrefEntity> hrefList) {
// validate
ApplicationEntity application = applicationService.getById(applicationId);
T.VerifyUtil.is(application).notNull(RCode.APP_NOT_EXIST);
for (ApplicationHrefEntity href : hrefList) {
T.VerifyUtil.is(href).notNull()
.and(href.getName()).notEmpty(RCode.NAME_CANNOT_EMPTY)
.and(href.getUrl()).notEmpty(RCode.PARAM_CANNOT_EMPTY);
href.setApplicationId(applicationId);
}
// save or update batch
List<ApplicationHrefEntity> entityList = hrefService.updateBatchHref(hrefList);
List<Map<String, String>> records = entityList.stream()
.map(entity -> Map.of("id", entity.getId()))
.collect(Collectors.toList());
return R.ok().putData("records", records); return R.ok().putData("records", records);
} }
@DeleteMapping("/{applicationId}/href") @PostMapping("/{workspaceId}/branch/{branchName}/application")
public R deleteHref(@PathVariable String applicationId, @RequestParam String[] ids) { public synchronized R newApplication(@PathVariable("workspaceId") String workspaceId,
// remove @PathVariable("branchName") String branchName,
hrefService.remove(new LambdaQueryWrapper<ApplicationHrefEntity>() @RequestBody Map<String, String> body) {
.eq(ApplicationHrefEntity::getApplicationId, applicationId) String applicationName = T.MapUtil.getStr(body, "name");
.in(ApplicationHrefEntity::getId, T.ListUtil.of(ids))); T.VerifyUtil.is(applicationName).notEmpty(RCode.PARAM_CANNOT_EMPTY);
applicationService.newApplication(workspaceId, branchName, applicationName);
return R.ok();
}
@PostMapping("/{workspaceId}/branch/{branchName}/application/{applicationName}/rename")
public R renameApplication(@PathVariable("workspaceId") String workspaceId,
@PathVariable("branchName") String branchName,
@PathVariable("applicationName") String applicationName,
@RequestBody Map<String, String> body) {
String newName = T.MapUtil.getStr(body, "name");
T.VerifyUtil.is(newName).notEmpty(RCode.PARAM_CANNOT_EMPTY);
// url decode
applicationName = T.URLUtil.decode(applicationName, StandardCharsets.UTF_8);
applicationService.renameApplication(workspaceId, branchName, applicationName, newName);
return R.ok();
}
@PostMapping("/{workspaceId}/branch/{branchName}/application/commit")
public synchronized R updateApplication(@PathVariable("workspaceId") String workspaceId,
@PathVariable("branchName") String branchName,
@RequestBody Map<String, Object> body) {
String lastCommitId = T.MapUtil.getStr(body, "lastCommitId");
String message = T.MapUtil.getStr(body, "message");
T.VerifyUtil.is(lastCommitId).notEmpty(RCode.PARAM_CANNOT_EMPTY)
.and(message).notEmpty(RCode.PARAM_CANNOT_EMPTY);
List<Map<String, String>> files = T.MapUtil.get(body, "files", List.class, T.ListUtil.list(true));
if (T.CollUtil.isEmpty(files)) {
return R.ok();
}
for (Map<String, String> file : files) {
String action = T.MapUtil.getStr(file, "action");
String path = T.MapUtil.getStr(file, "path");
if (T.StrUtil.hasEmpty(action, path)) {
return R.error(RCode.PARAM_CANNOT_EMPTY);
}
if (T.StrUtil.equalsAny(action, "create", "update")) {
String content = T.MapUtil.getStr(file, "content");
T.VerifyUtil.is(content).notEmpty(RCode.PARAM_CANNOT_EMPTY);
}
}
applicationService.updateApplication(workspaceId, branchName, lastCommitId, message, files);
return R.ok();
}
@DeleteMapping("/{workspaceId}/branch/{branchName}/application/{applicationName}")
public synchronized R deleteApplication(@PathVariable("workspaceId") String workspaceId,
@PathVariable("branchName") String branchName,
@PathVariable("applicationName") String applicationName) {
applicationService.deleteApplication(workspaceId, branchName, applicationName);
return R.ok(); return R.ok();
} }
@PostMapping("/import") @GetMapping("/{workspaceId}/branch/{branchName}/application/{applicationName}/commit")
public R importApplication(@RequestParam String workspaceId, public R listApplicationCommit(@PathVariable("workspaceId") String workspaceId,
@PathVariable("branchName") String branchName,
@PathVariable("applicationName") String applicationName,
@RequestParam(required = false) String file) {
List<Map<Object, Object>> records = applicationService.listApplicationCommit(workspaceId, branchName, applicationName, file);
return R.ok().putData("records", records);
}
@GetMapping("/{workspaceId}/branch/{branchName}/application/{applicationName}/commit/{commitId}/content")
public R infoApplicationFileContent(@PathVariable("workspaceId") String workspaceId,
@PathVariable("branchName") String branchName,
@PathVariable("applicationName") String applicationName,
@PathVariable("commitId") String commitId,
@RequestParam(required = true) String file) {
Map<Object, Object> record = applicationService.infoApplicationFileContent(workspaceId, branchName, applicationName, commitId, file);
return R.ok().putData("record", record);
}
@PostMapping("/{workspaceId}/branch/{branchName}/application/import")
public synchronized R importApplication(@PathVariable("workspaceId") String workspaceId,
@PathVariable("branchName") String branchName,
@RequestParam(defaultValue = "tsg2402") String format, @RequestParam(defaultValue = "tsg2402") String format,
@RequestParam(value = "files") List<MultipartFile> fileList) { @RequestParam(value = "files") List<MultipartFile> fileList) {
// validate // validate
@@ -292,25 +155,36 @@ public class ApplicationController {
throw new ASWException(RCode.APP_IMPORT_FILE_FORMAT_ERROR); throw new ASWException(RCode.APP_IMPORT_FILE_FORMAT_ERROR);
} }
// 名称重复校验
List<Object> applicationList = dataList.stream()
.map(entries -> entries.getJSONArray("applications"))
.flatMap(Collection::stream)
.collect(Collectors.toList());
long distinctNameCount = applicationList.stream()
.map(obj -> ((JSONObject) obj).getStr("app_name"))
.distinct()
.count();
long size = applicationList.size();
if (T.ObjectUtil.notEqual(size, distinctNameCount)) {
throw new ASWException(RCode.APP_DUPLICATE_RECORD);
}
// import // import
List<ApplicationEntity> entityList = applicationService.importAppByFormat(workspaceId, format, dataList); List<ApplicationEntity> entityList = applicationService.importAppByFormat(workspaceId, branchName, format, dataList);
List<Map<String, String>> records = entityList.stream() List<Map<String, String>> records = entityList.stream()
.map(entity -> Map.of("id", entity.getId())) .map(entity -> Map.of("name", entity.getName()))
.collect(Collectors.toList()); .collect(Collectors.toList());
return R.ok().putData("records", records); return R.ok().putData("records", records);
} }
@GetMapping("/export") @GetMapping("/{workspaceId}/branch/{branchName}/application/export")
public void exportApplication(@RequestParam String workspaceId, public void exportApplication(@PathVariable("workspaceId") String workspaceId,
@RequestParam String ids, @PathVariable("branchName") String branchName,
@RequestParam String names,
@RequestParam(defaultValue = "tsg2402") String format, @RequestParam(defaultValue = "tsg2402") String format,
HttpServletResponse response) throws IOException { HttpServletResponse response) throws IOException {
// validate // validate
List<ApplicationEntity> appList = applicationService.list( List<ApplicationEntity> appList = applicationService.listApplication(workspaceId, branchName, T.StrUtil.splitToArray(names, ","));
new LambdaQueryWrapper<ApplicationEntity>()
.eq(ApplicationEntity::getWorkspaceId, workspaceId)
.in(ApplicationEntity::getId, T.ListUtil.of(ids.split(",")))
);
T.VerifyUtil.is(appList).notEmpty(RCode.APP_NOT_EXIST); T.VerifyUtil.is(appList).notEmpty(RCode.APP_NOT_EXIST);
// format // format

View File

@@ -0,0 +1,91 @@
package net.geedge.asw.module.app.controller;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import jakarta.servlet.http.HttpServletResponse;
import net.geedge.asw.common.util.*;
import net.geedge.asw.module.app.entity.ApplicationReleaseEntity;
import net.geedge.asw.module.app.service.IApplicationReleaseService;
import net.geedge.asw.module.workspace.entity.WorkspaceEntity;
import net.geedge.asw.module.workspace.service.IWorkspaceService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.MediaType;
import org.springframework.web.bind.annotation.*;
import java.io.IOException;
import java.util.HashMap;
import java.util.Map;
@RestController
@RequestMapping("/api/v1/workspace")
public class ApplicationReleaseController {
@Autowired
private IWorkspaceService workspaceService;
@Autowired
private IApplicationReleaseService releaseService;
@GetMapping("/{workspaceId}/release/{id}")
public R detail(@PathVariable("workspaceId") String workspaceId, @PathVariable("id") String id) {
ApplicationReleaseEntity record = releaseService.queryInfo(id);
return R.ok().putData("record", record);
}
@GetMapping("/{workspaceId}/release")
public R list(@PathVariable("workspaceId") String workspaceId, @RequestParam Map<String, Object> params) {
// workspaceId
params = T.MapUtil.defaultIfEmpty(params, new HashMap<>());
params.put("workspaceId", workspaceId);
Page page = releaseService.queryList(params);
return R.ok(page);
}
@PostMapping("/{workspaceId}/release")
public R add(@PathVariable("workspaceId") String workspaceId, @RequestBody Map<String, String> requestBody) {
String name = T.MapUtil.getStr(requestBody, "name", "");
String tagName = T.MapUtil.getStr(requestBody, "tagName", "");
String description = T.MapUtil.getStr(requestBody, "description", "");
if (T.StrUtil.hasEmpty(name, tagName)) {
throw new ASWException(RCode.PARAM_CANNOT_EMPTY);
}
ApplicationReleaseEntity record = releaseService.saveRelease(workspaceId, name, tagName, description);
return R.ok().putData("record", record);
}
@PutMapping("/{workspaceId}/release")
public R update(@PathVariable("workspaceId") String workspaceId, @RequestBody Map<String, String> requestBody) {
String id = T.MapUtil.getStr(requestBody, "id", "");
String name = T.MapUtil.getStr(requestBody, "name", "");
String description = T.MapUtil.getStr(requestBody, "description", "");
if (T.StrUtil.hasEmpty(id, name)) {
throw new ASWException(RCode.PARAM_CANNOT_EMPTY);
}
ApplicationReleaseEntity record = releaseService.updateRelease(id, name, description);
return R.ok().putData("record", record);
}
@DeleteMapping("/{workspaceId}/release/{id}")
public R delete(@PathVariable("workspaceId") String workspaceId, @PathVariable("id") String id) {
releaseService.removeRelease(id);
return R.ok();
}
@GetMapping("/{workspaceId}/release/{id}/file")
public void download(@PathVariable("workspaceId") String workspaceId,
@PathVariable("id") String id,
HttpServletResponse response) throws IOException {
ApplicationReleaseEntity release = releaseService.getById(id);
T.VerifyUtil.is(release).notNull(RCode.SYS_RECORD_NOT_FOUND);
WorkspaceEntity workspace = workspaceService.getById(workspaceId);
T.VerifyUtil.is(workspace).notNull(RCode.SYS_RECORD_NOT_FOUND);
String fileName = T.StrUtil.concat(true, workspace.getName(), "-", release.getTagName(), ".zip");
byte[] fileBytes = T.FileUtil.readBytes(T.FileUtil.file(release.getPath()));
ResponseUtil.downloadFile(response, MediaType.APPLICATION_OCTET_STREAM_VALUE, fileName, fileBytes);
}
}

View File

@@ -0,0 +1,111 @@
package net.geedge.asw.module.app.controller;
import net.geedge.asw.common.util.ASWException;
import net.geedge.asw.common.util.R;
import net.geedge.asw.common.util.RCode;
import net.geedge.asw.common.util.T;
import net.geedge.asw.module.app.service.IBranchService;
import net.geedge.asw.module.app.util.JGitUtils;
import net.geedge.asw.module.workspace.service.IWorkspaceService;
import org.eclipse.jgit.lib.Repository;
import org.eclipse.jgit.revwalk.RevCommit;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.*;
import java.io.File;
import java.io.IOException;
import java.util.List;
import java.util.Map;
@RestController
@RequestMapping("/api/v1/workspace")
public class BranchController {
@Autowired
private IBranchService branchService;
@Autowired
private IWorkspaceService workspaceService;
@GetMapping("/{workspaceId}/branch")
public R listBranch(@PathVariable("workspaceId") String workspaceId,
@RequestParam(value = "search", required = false) String search) {
List<Map<Object, Object>> list = branchService.listBranch(workspaceId, search);
return R.ok().putData("records", list);
}
@GetMapping("/{workspaceId}/branch/{branchName}")
public R infoBranch(@PathVariable("workspaceId") String workspaceId, @PathVariable("branchName") String branchName) {
Map<Object, Object> record = branchService.infoBranch(workspaceId, branchName);
return R.ok().putData("record", record);
}
@PostMapping("/{workspaceId}/branch")
public synchronized R newBranch(@PathVariable("workspaceId") String workspaceId, @RequestBody Map<String, String> requestBody) {
String branch = T.MapUtil.getStr(requestBody, "branch", "");
String ref = T.MapUtil.getStr(requestBody, "ref", "");
if (T.StrUtil.hasEmpty(branch, ref)) {
throw new ASWException(RCode.PARAM_CANNOT_EMPTY);
}
Map<Object, Object> record = branchService.newBranch(workspaceId, branch, ref);
return R.ok().putData("record", record);
}
@DeleteMapping("/{workspaceId}/branch/{branchName}")
public synchronized R deleteBranch(@PathVariable("workspaceId") String workspaceId, @PathVariable("branchName") String branchName) {
branchService.deleteBranch(workspaceId, branchName);
return R.ok();
}
@GetMapping("/{workspaceId}/branch/{branchName}/commits")
public R listBranchCommit(@PathVariable("workspaceId") String workspaceId,
@PathVariable("branchName") String branchName,
@RequestParam(required = false) String path,
@RequestParam(required = false) Integer current,
@RequestParam(required = false) Integer size) {
List<Map<Object, Object>> records = branchService.listBranchCommit(workspaceId, branchName, path);
long total = T.CollUtil.size(records);
if (null != current && null != size) {
int fromIndex = (current - 1) * size;
int toIndex = Math.min(fromIndex + size, records.size());
records = records.subList(fromIndex, toIndex);
}
return R.ok().putData("records", records).put("total", total);
}
@GetMapping({
"/{workspaceId}/branch/{branchName}/commit/{commitId}/diff",
"/{workspaceId}/branch/{branchName}/commit/{commitId}/diff/{oldCommitId}"
})
public R branchCommitDiff(@PathVariable("workspaceId") String workspaceId,
@PathVariable("branchName") String branchName,
@PathVariable("commitId") String newCommitId,
@PathVariable(value = "oldCommitId", required = false) String oldCommitId) throws IOException {
File gitDir = workspaceService.getGitDir(workspaceId);
try (Repository repository = JGitUtils.openRepository(gitDir)) {
if (T.StrUtil.isEmpty(oldCommitId)) {
// oldCommitId 为空默认对比 parent commitId
RevCommit parentCommit = JGitUtils.getParentCommit(repository, branchName, newCommitId);
oldCommitId = parentCommit.getName();
}
RevCommit oldCommit = JGitUtils.infoCommit(repository, oldCommitId);
RevCommit newCommit = JGitUtils.infoCommit(repository, newCommitId);
List<Map<Object, Object>> diffList = JGitUtils.getDiffFileListInCommits(repository, newCommitId, oldCommitId);
Map<Object, Object> record = T.MapUtil.builder()
.put("oldBranch", branchName)
.put("newBranch", branchName)
.put("oldCommitId", oldCommitId)
.put("newCommitId", newCommitId)
.put("oldCommit", JGitUtils.buildAswCommitInfo(oldCommit))
.put("newCommit", JGitUtils.buildAswCommitInfo(newCommit))
.put("files", diffList)
.build();
return R.ok().putData("record", record);
}
}
}

View File

@@ -1,52 +0,0 @@
package net.geedge.asw.module.app.controller;
import net.geedge.asw.common.util.ASWException;
import net.geedge.asw.common.util.R;
import net.geedge.asw.common.util.RCode;
import net.geedge.asw.common.util.T;
import net.geedge.asw.module.app.service.IGitService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.*;
import java.util.List;
import java.util.Map;
@RestController
@RequestMapping("/api/v1/workspace")
public class GitController {
@Autowired
private IGitService gitService;
@GetMapping("/{workspaceId}/branch")
public R listBranch(@PathVariable("workspaceId") String workspaceId,
@RequestParam(value = "search", required = false) String search) {
List<Map<Object, Object>> list = gitService.listBranch(workspaceId, search);
return R.ok().putData("records", list);
}
@GetMapping("/{workspaceId}/branch/{branchName}")
public R infoBranch(@PathVariable("workspaceId") String workspaceId, @PathVariable("branchName") String branchName) {
Map<Object, Object> record = gitService.infoBranch(workspaceId, branchName);
return R.ok().putData("record", record);
}
@PostMapping("/{workspaceId}/branch")
public R newBranch(@PathVariable("workspaceId") String workspaceId, @RequestBody Map<String, String> requestBody) {
String branch = T.MapUtil.getStr(requestBody, "branch", "");
String ref = T.MapUtil.getStr(requestBody, "ref", "");
if (T.StrUtil.hasEmpty(branch, ref)) {
throw new ASWException(RCode.PARAM_CANNOT_EMPTY);
}
Map<Object, Object> record = gitService.newBranch(workspaceId, branch, ref);
return R.ok().putData("record", record);
}
@DeleteMapping("/{workspaceId}/branch/{branchName}")
public R deleteBranch(@PathVariable("workspaceId") String workspaceId, @PathVariable("branchName") String branchName) {
gitService.deleteBranch(workspaceId, branchName);
return R.ok();
}
}

View File

@@ -0,0 +1,109 @@
package net.geedge.asw.module.app.controller;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import net.geedge.asw.common.util.R;
import net.geedge.asw.common.util.RCode;
import net.geedge.asw.common.util.T;
import net.geedge.asw.module.app.entity.ApplicationMergeEntity;
import net.geedge.asw.module.app.service.IApplicationMergeService;
import net.geedge.asw.module.app.service.impl.ApplicationMergeServiceImpl.MergeRequestStatus;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.*;
import java.io.IOException;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
@RestController
@RequestMapping("/api/v1/workspace")
public class MergeRequestController {
@Autowired
private IApplicationMergeService applicationMergeService;
@GetMapping("/{workspaceId}/mr")
public R listMr(@PathVariable("workspaceId") String workspaceId, @RequestParam Map<String, Object> params) {
// workspaceId
params = T.MapUtil.defaultIfEmpty(params, new HashMap<>());
params.put("workspaceId", workspaceId);
Page page = applicationMergeService.queryList(params);
return R.ok(page);
}
@GetMapping("/{workspaceId}/mr/{mrId}")
public R infoMr(@PathVariable("mrId") String mrId) {
ApplicationMergeEntity record = applicationMergeService.queryInfo(mrId);
return R.ok().putData("record", record);
}
@GetMapping("/{workspaceId}/mr/{mrId}/commits")
public R listMrCommit(@PathVariable("mrId") String mrId) throws IOException {
List<Map<Object, Object>> records = applicationMergeService.listMrCommit(mrId);
return R.ok().putData("records", records);
}
@GetMapping("/{workspaceId}/mr/{mrId}/diffs")
public R mrDiff(@PathVariable("mrId") String mrId) {
Map<Object, Object> record = applicationMergeService.mrCommitDiff(mrId);
return R.ok().putData("record", record);
}
@GetMapping("/{workspaceId}/mr/{mrId}/conflicts")
public R getMrConflictList(@PathVariable("mrId") String mrId) throws IOException {
Map<Object, Object> record = applicationMergeService.getMrConflictList(mrId);
return R.ok().putData("record", record);
}
@PostMapping("/{workspaceId}/mr/{mrId}/conflicts")
public R resolveMrConflicts(@PathVariable("mrId") String mrId, @RequestBody Map<String, Object> body) throws IOException{
String commitId = T.MapUtil.getStr(body, "commitId");
String commitMessage = T.MapUtil.getStr(body, "commitMessage");
List<Map<String, String>> files = T.MapUtil.get(body, "files", List.class, new ArrayList());
T.VerifyUtil.is(commitId).notEmpty(RCode.PARAM_CANNOT_EMPTY)
.and(commitMessage).notEmpty(RCode.PARAM_CANNOT_EMPTY)
.and(files).notEmpty(RCode.PARAM_CANNOT_EMPTY);
for (Map<String, String> m : files) {
String path = T.MapUtil.getStr(m, "path");
String content = T.MapUtil.getStr(m, "content");
T.VerifyUtil.is(path).notEmpty(RCode.PARAM_CANNOT_EMPTY)
.and(content).notEmpty(RCode.PARAM_CANNOT_EMPTY);
}
applicationMergeService.resolveMrConflicts(mrId, body);
return R.ok();
}
@PostMapping("/{workspaceId}/mr")
public R newMr(@PathVariable("workspaceId") String workspaceId, @RequestBody ApplicationMergeEntity entity) throws IOException {
T.VerifyUtil.is(entity).notNull()
.and(entity.getTitle()).notEmpty(RCode.PARAM_CANNOT_EMPTY)
.and(entity.getTargetBranch()).notEmpty(RCode.PARAM_CANNOT_EMPTY)
.and(entity.getTargetBranch()).notEmpty(RCode.PARAM_CANNOT_EMPTY);
// workspaceId
entity.setWorkspaceId(workspaceId);
ApplicationMergeEntity record = applicationMergeService.newMr(entity);
return R.ok().putData("record", record);
}
@PutMapping("/{workspaceId}/mr/{mrId}")
public synchronized R mergeMr(@PathVariable("mrId") String mrId, @RequestBody(required = false) Map<String, Object> body) {
String action = T.MapUtil.getStr(body, "action");
T.VerifyUtil.is(action).notEmpty(RCode.PARAM_CANNOT_EMPTY);
ApplicationMergeEntity record = applicationMergeService.mergeMr(mrId, action);
if(T.BooleanUtil.and(
T.ObjectUtil.equals("merge", action),
T.ObjectUtil.notEqual(MergeRequestStatus.MERGED.toString(), record.getStatus())
)){
return R.error(RCode.GIT_MERGE_FAILED).putData("record", record);
}
return R.ok().putData("record", record);
}
}

View File

@@ -50,6 +50,16 @@ public class PackageController {
return R.ok().putData("record", entity); return R.ok().putData("record", entity);
} }
@PutMapping("/{workspaceId}/package")
public R update(@PathVariable(value = "workspaceId", required = true) String workspaceId,
@RequestParam(value = "packageId", required = true) String packageId,
@RequestParam(value = "description", required = false) String description,
@RequestParam(value = "name", required = false) String name) {
PackageEntity entity = packageService.updatePackage(workspaceId, packageId, name, description);
return R.ok().putData("record", entity);
}
@DeleteMapping("/{workspaceId}/package") @DeleteMapping("/{workspaceId}/package")
public R delete(String[] ids) { public R delete(String[] ids) {
T.VerifyUtil.is(ids).notEmpty(); T.VerifyUtil.is(ids).notEmpty();
@@ -65,7 +75,7 @@ public class PackageController {
T.VerifyUtil.is(entity).notNull(RCode.SYS_RECORD_NOT_FOUND); T.VerifyUtil.is(entity).notNull(RCode.SYS_RECORD_NOT_FOUND);
File pkgFile = T.FileUtil.file(entity.getPath()); File pkgFile = T.FileUtil.file(entity.getPath());
ResponseUtil.downloadFile(response, MediaType.APPLICATION_OCTET_STREAM_VALUE, entity.getName(), T.FileUtil.readBytes(pkgFile)); ResponseUtil.downloadFile(response, MediaType.APPLICATION_OCTET_STREAM_VALUE, pkgFile.getName(), T.FileUtil.readBytes(pkgFile));
} }
} }

View File

@@ -0,0 +1,52 @@
package net.geedge.asw.module.app.controller;
import net.geedge.asw.common.util.ASWException;
import net.geedge.asw.common.util.R;
import net.geedge.asw.common.util.RCode;
import net.geedge.asw.common.util.T;
import net.geedge.asw.module.app.service.ITagService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.*;
import java.util.List;
import java.util.Map;
@RestController
@RequestMapping("/api/v1/workspace")
public class TagController {
@Autowired
private ITagService tagService;
@GetMapping("/{workspaceId}/tag/{tagName}")
public R infoTag(@PathVariable("workspaceId") String workspaceId, @PathVariable("tagName") String name) {
Map<Object, Object> record = tagService.infoTag(workspaceId, name);
return R.ok().putData("record", record);
}
@GetMapping("/{workspaceId}/tag")
public R listTag(@PathVariable("workspaceId") String workspaceId,
@RequestParam(value = "search", required = false) String search) {
List<Map<Object, Object>> list = tagService.listTag(workspaceId, search);
return R.ok().putData("records", list);
}
@PostMapping("/{workspaceId}/tag")
public synchronized R newTag(@PathVariable("workspaceId") String workspaceId, @RequestBody Map<String, String> requestBody) {
String name = T.MapUtil.getStr(requestBody, "name", "");
String ref = T.MapUtil.getStr(requestBody, "ref", "");
String description = T.MapUtil.getStr(requestBody, "description", "");
if (T.StrUtil.hasEmpty(name, ref)) {
throw new ASWException(RCode.PARAM_CANNOT_EMPTY);
}
Map<Object, Object> record = tagService.newTag(workspaceId, name, ref, description);
return R.ok().putData("record", record);
}
@DeleteMapping("/{workspaceId}/tag/{tagName}")
public synchronized R deleteTag(@PathVariable("workspaceId") String workspaceId, @PathVariable("tagName") String name) {
tagService.deleteTag(workspaceId, name);
return R.ok();
}
}

View File

@@ -2,14 +2,15 @@ package net.geedge.asw.module.app.dao;
import com.baomidou.mybatisplus.core.mapper.BaseMapper; import com.baomidou.mybatisplus.core.mapper.BaseMapper;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page; import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import net.geedge.asw.module.app.entity.ApplicationEntity; import net.geedge.asw.module.app.entity.ApplicationMergeEntity;
import org.apache.ibatis.annotations.Mapper; import org.apache.ibatis.annotations.Mapper;
import java.util.List; import java.util.List;
import java.util.Map; import java.util.Map;
@Mapper
public interface ApplicationDao extends BaseMapper<ApplicationEntity>{
List<ApplicationEntity> queryList(Page page, Map<String, Object> params); @Mapper
public interface ApplicationMergeDao extends BaseMapper<ApplicationMergeEntity> {
List<ApplicationMergeEntity> queryList(Page page, Map<String, Object> params);
} }

View File

@@ -0,0 +1,17 @@
package net.geedge.asw.module.app.dao;
import com.baomidou.mybatisplus.core.mapper.BaseMapper;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import net.geedge.asw.module.app.entity.ApplicationReleaseEntity;
import org.apache.ibatis.annotations.Mapper;
import java.util.List;
import java.util.Map;
@Mapper
public interface ApplicationReleaseDao extends BaseMapper<ApplicationReleaseEntity> {
List<ApplicationReleaseEntity> queryList(Page page, Map<String, Object> params);
}

View File

@@ -1,67 +1,26 @@
package net.geedge.asw.module.app.entity; package net.geedge.asw.module.app.entity;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableField; import com.baomidou.mybatisplus.annotation.TableField;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
import com.baomidou.mybatisplus.extension.handlers.JacksonTypeHandler; import com.baomidou.mybatisplus.extension.handlers.JacksonTypeHandler;
import lombok.Data; import lombok.Data;
import net.geedge.asw.module.sys.entity.SysUserEntity;
import java.util.List;
@Data @Data
@TableName(value = "application", autoResultMap = true)
public class ApplicationEntity { public class ApplicationEntity {
@TableId(type = IdType.ASSIGN_UUID)
private String id; private String id;
private String name; private String name;
private String longName;
private String tags; private String developer;
private String packageName;
private String website; private String website;
private String description;
private String provider; @TableField(typeHandler = JacksonTypeHandler.class)
private Object packageName;
@TableField(typeHandler = JacksonTypeHandler.class) @TableField(typeHandler = JacksonTypeHandler.class)
private Object properties; private Object properties;
private String status; @TableField(typeHandler = JacksonTypeHandler.class)
private Object signature;
private String description;
private Long createTimestamp;
private Long updateTimestamp;
private String createUserId;
private String updateUserId;
private String workspaceId;
private Integer opVersion;
@TableField(exist = false)
private SysUserEntity createUser;
@TableField(exist = false)
private SysUserEntity updateUser;
@TableField(exist = false)
private ApplicationSignatureEntity signature;
@TableField(exist = false)
private ApplicationNoteEntity note;
@TableField(exist = false)
private List<ApplicationAttachmentEntity> attatchments;
@TableField(exist = false)
private List<ApplicationHrefEntity> hrefs;
} }

View File

@@ -0,0 +1,37 @@
package net.geedge.asw.module.app.entity;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableField;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
import lombok.Data;
import net.geedge.asw.module.sys.entity.SysUserEntity;
@Data
@TableName(value = "application_merge", autoResultMap = true)
public class ApplicationMergeEntity {
@TableId(type = IdType.ASSIGN_UUID)
private String id;
private String sourceBranch;
private String targetBranch;
private String startCommitId;
private String endCommitId;
private String title;
private Integer removeSourceBranch = 0;
private String description;
private String status;
private String message;
private Long createTimestamp;
private String createUserId;
private String workspaceId;
@TableField(exist = false)
private SysUserEntity createUser;
@TableField(exist = false)
private Object commit;
}

View File

@@ -0,0 +1,40 @@
package net.geedge.asw.module.app.entity;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableField;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
import lombok.Data;
import net.geedge.asw.module.sys.entity.SysUserEntity;
@Data
@TableName(value = "application_release", autoResultMap = true)
public class ApplicationReleaseEntity {
@TableId(type = IdType.ASSIGN_UUID)
private String id;
private String name;
private String tagName;
private String path;
private String description;
private String createUserId;
private Long createTimestamp;
private String updateUserId;
private Long updateTimestamp;
private String workspaceId;
@TableField(exist = false)
private SysUserEntity createUser;
@TableField(exist = false)
private SysUserEntity updateUser;
@TableField(exist = false)
private Object commit;
}

View File

@@ -4,6 +4,7 @@ import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableField; import com.baomidou.mybatisplus.annotation.TableField;
import com.baomidou.mybatisplus.annotation.TableId; import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName; import com.baomidou.mybatisplus.annotation.TableName;
import com.fasterxml.jackson.annotation.JsonIgnore;
import lombok.Data; import lombok.Data;
import net.geedge.asw.module.sys.entity.SysUserEntity; import net.geedge.asw.module.sys.entity.SysUserEntity;
@@ -19,6 +20,8 @@ public class PackageEntity {
private String platform; private String platform;
private String version; private String version;
private String identifier; private String identifier;
@JsonIgnore
private String path; private String path;
private Long size; private Long size;

View File

@@ -0,0 +1,29 @@
package net.geedge.asw.module.app.service;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import com.baomidou.mybatisplus.extension.service.IService;
import net.geedge.asw.module.app.entity.ApplicationMergeEntity;
import java.io.IOException;
import java.util.List;
import java.util.Map;
public interface IApplicationMergeService extends IService<ApplicationMergeEntity> {
ApplicationMergeEntity queryInfo(String mrId);
Page queryList(Map<String, Object> params);
ApplicationMergeEntity newMr(ApplicationMergeEntity entity) throws IOException;
List<Map<Object, Object>> listMrCommit(String mrId) throws IOException;
Map<Object, Object> mrCommitDiff(String mrId);
Map<Object, Object> getMrConflictList(String mrId) throws IOException;
void resolveMrConflicts(String mrId, Map<String, Object> body) throws IOException;
ApplicationMergeEntity mergeMr(String mrId, String action);
}

View File

@@ -0,0 +1,23 @@
package net.geedge.asw.module.app.service;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import com.baomidou.mybatisplus.extension.service.IService;
import net.geedge.asw.module.app.entity.ApplicationReleaseEntity;
import java.util.Map;
public interface IApplicationReleaseService extends IService<ApplicationReleaseEntity> {
ApplicationReleaseEntity queryInfo(String id);
Page queryList(Map<String, Object> params);
ApplicationReleaseEntity saveRelease(String workspaceId, String name, String tagName, String description);
ApplicationReleaseEntity updateRelease(String id, String name, String description);
void removeRelease(String id);
void removeRelease(String workspaceId, String tagName);
}

View File

@@ -1,29 +1,33 @@
package net.geedge.asw.module.app.service; package net.geedge.asw.module.app.service;
import cn.hutool.json.JSONObject; import cn.hutool.json.JSONObject;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import com.baomidou.mybatisplus.extension.service.IService;
import net.geedge.asw.module.app.entity.ApplicationEntity; import net.geedge.asw.module.app.entity.ApplicationEntity;
import java.util.List; import java.util.List;
import java.util.Map; import java.util.Map;
public interface IApplicationService extends IService<ApplicationEntity>{ public interface IApplicationService {
ApplicationEntity detail(String id, String workspaceId); Map<Object, Object> infoApplication(String workspaceId, String branch, String name);
Page queryList(Map<String, Object> params); List<ApplicationEntity> listApplication(String workspaceId, String branch, String... names);
ApplicationEntity saveApplication(ApplicationEntity entity); List<Map<Object, Object>> listApplication(String workspaceId, String branch, String q);
ApplicationEntity updateApplication(ApplicationEntity entity); void newApplication(String workspaceId, String branch, String name);
ApplicationEntity updateBasic(ApplicationEntity entity); void renameApplication(String workspaceId, String branch, String oldName, String newName);
void removeApplication(List<String> ids); void deleteApplication(String workspaceId, String branch, String name);
void updateApplication(String workspaceId, String branch, String lastCommitId, String message, List<Map<String, String>> updateContent);
List<Map<Object, Object>> listApplicationCommit(String workspaceId, String branch, String name, String file);
Map<Object, Object> infoApplicationFileContent(String workspaceId, String branch, String name, String commitId, String file);
byte[] exportAppByFormat(List<ApplicationEntity> appList, String format); byte[] exportAppByFormat(List<ApplicationEntity> appList, String format);
List<ApplicationEntity> importAppByFormat(String workspaceId, String format, List<JSONObject> dataList); List<ApplicationEntity> importAppByFormat(String workspaceId, String branch, String format, List<JSONObject> dataList);
} }

View File

@@ -0,0 +1,18 @@
package net.geedge.asw.module.app.service;
import java.util.List;
import java.util.Map;
public interface IBranchService {
List<Map<Object, Object>> listBranch(String workspaceId, String search);
Map<Object, Object> infoBranch(String workspaceId, String branch);
Map<Object, Object> newBranch(String workspaceId, String name, String ref);
void deleteBranch(String workspaceId, String branch);
List<Map<Object, Object>> listBranchCommit(String workspaceId, String branch, String path);
}

View File

@@ -1,16 +0,0 @@
package net.geedge.asw.module.app.service;
import java.util.List;
import java.util.Map;
public interface IGitService {
List<Map<Object, Object>> listBranch(String workspaceId, String search);
Map<Object, Object> infoBranch(String workspaceId, String branchName);
Map<Object, Object> newBranch(String workspaceId, String branchName, String ref);
void deleteBranch(String workspaceId, String branchName);
}

View File

@@ -17,4 +17,6 @@ public interface IPackageService extends IService<PackageEntity>{
PackageEntity savePackage(String workspaceId, String description, Resource fileResource); PackageEntity savePackage(String workspaceId, String description, Resource fileResource);
void removePackage(List<String> ids); void removePackage(List<String> ids);
PackageEntity updatePackage(String workspaceId, String packageId, String name, String description);
} }

View File

@@ -12,4 +12,8 @@ public interface ITSGApplicationService {
List<ApplicationEntity> tsg2402ToAsw(String workspaceId, List<JSONObject> dataList); List<ApplicationEntity> tsg2402ToAsw(String workspaceId, List<JSONObject> dataList);
Map<Object, Object> aswToTsg2410(List<ApplicationEntity> appList);
List<ApplicationEntity> tsg2410ToAsw(String workspaceId, List<JSONObject> dataList);
} }

View File

@@ -0,0 +1,16 @@
package net.geedge.asw.module.app.service;
import java.util.List;
import java.util.Map;
public interface ITagService {
Map<Object, Object> infoTag(String workspaceId, String name);
List<Map<Object, Object>> listTag(String workspaceId, String search);
Map<Object, Object> newTag(String workspaceId, String name, String branch, String message);
void deleteTag(String workspaceId, String name);
}

View File

@@ -30,41 +30,42 @@ public class ApplicationAttachmentServiceImpl extends ServiceImpl<ApplicationAtt
private static final Log log = Log.get(); private static final Log log = Log.get();
@Autowired // @Autowired
private IApplicationService applicationService; // private IApplicationService applicationService;
@Override @Override
public ApplicationAttachmentEntity saveAttachment(Resource fileResource, String applicationId) { public ApplicationAttachmentEntity saveAttachment(Resource fileResource, String applicationId) {
ApplicationEntity app = applicationService.getById(applicationId); // ApplicationEntity app = applicationService.getById(applicationId);
ApplicationAttachmentEntity entity = new ApplicationAttachmentEntity(); // ApplicationAttachmentEntity entity = new ApplicationAttachmentEntity();
try { // try {
entity.setName(fileResource.getFilename()); // entity.setName(fileResource.getFilename());
entity.setCreateTimestamp(System.currentTimeMillis()); // entity.setCreateTimestamp(System.currentTimeMillis());
entity.setCreateUserId(StpUtil.getLoginIdAsString()); // entity.setCreateUserId(StpUtil.getLoginIdAsString());
entity.setApplicationId(applicationId); // entity.setApplicationId(applicationId);
//
// path // // path
File destination = T.FileUtil.file(T.WebPathUtil.getRootPath(), app.getId(), fileResource.getFilename()); // File destination = T.FileUtil.file(T.WebPathUtil.getRootPath(), app.getId(), fileResource.getFilename());
FileUtils.copyInputStreamToFile(fileResource.getInputStream(), destination); // FileUtils.copyInputStreamToFile(fileResource.getInputStream(), destination);
entity.setPath(destination.getPath()); // entity.setPath(destination.getPath());
//
// 根据文件 applicationId path 判断是否已上存在,存在则响应当前实体 // // 根据文件 applicationId path 判断是否已上存在,存在则响应当前实体
ApplicationAttachmentEntity attachment = this.getOne(new LambdaQueryWrapper<ApplicationAttachmentEntity>() // ApplicationAttachmentEntity attachment = this.getOne(new LambdaQueryWrapper<ApplicationAttachmentEntity>()
.eq(ApplicationAttachmentEntity::getApplicationId, applicationId) // .eq(ApplicationAttachmentEntity::getApplicationId, applicationId)
.eq(ApplicationAttachmentEntity::getPath, destination.getPath())); // .eq(ApplicationAttachmentEntity::getPath, destination.getPath()));
if (T.ObjectUtil.isNotNull(attachment)) { // if (T.ObjectUtil.isNotNull(attachment)) {
return attachment; // return attachment;
} // }
//
// save // // save
this.save(entity); // this.save(entity);
//
} catch (IOException e) { // } catch (IOException e) {
log.error(e, "[saveAttachment] [error] [applicationId: {}]", applicationId); // log.error(e, "[saveAttachment] [error] [applicationId: {}]", applicationId);
throw new ASWException(RCode.ERROR); // throw new ASWException(RCode.ERROR);
} // }
return entity; // return entity;
return null;
} }
@Override @Override

View File

@@ -0,0 +1,414 @@
package net.geedge.asw.module.app.service.impl;
import cn.dev33.satoken.stp.StpUtil;
import cn.hutool.core.util.StrUtil;
import cn.hutool.log.Log;
import com.baomidou.mybatisplus.core.conditions.update.LambdaUpdateWrapper;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;
import net.geedge.asw.common.config.Query;
import net.geedge.asw.common.util.ASWException;
import net.geedge.asw.common.util.RCode;
import net.geedge.asw.common.util.T;
import net.geedge.asw.module.app.dao.ApplicationMergeDao;
import net.geedge.asw.module.app.entity.ApplicationMergeEntity;
import net.geedge.asw.module.app.service.IApplicationMergeService;
import net.geedge.asw.module.app.service.IBranchService;
import net.geedge.asw.module.app.util.JGitUtils;
import net.geedge.asw.module.sys.entity.SysUserEntity;
import net.geedge.asw.module.sys.service.ISysUserService;
import net.geedge.asw.module.workspace.service.IWorkspaceService;
import org.eclipse.jgit.api.errors.GitAPIException;
import org.eclipse.jgit.lib.ObjectId;
import org.eclipse.jgit.lib.Repository;
import org.eclipse.jgit.revwalk.RevCommit;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import java.io.File;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
@Service
public class ApplicationMergeServiceImpl extends ServiceImpl<ApplicationMergeDao, ApplicationMergeEntity> implements IApplicationMergeService {
private final static Log log = Log.get();
@Autowired
private ISysUserService userService;
@Autowired
private IBranchService branchService;
@Autowired
private IWorkspaceService workspaceService;
@Override
public ApplicationMergeEntity queryInfo(String mrId) {
ApplicationMergeEntity entity = this.getById(mrId);
T.VerifyUtil.is(entity).notEmpty(RCode.SYS_RECORD_NOT_FOUND);
SysUserEntity user = userService.getById(entity.getCreateUserId());
entity.setCreateUser(user);
return entity;
}
@Override
public Page queryList(Map<String, Object> params) {
Page page = new Query(ApplicationMergeEntity.class).getPage(params);
List<ApplicationMergeEntity> list = this.getBaseMapper().queryList(page, params);
page.setRecords(list);
return page;
}
@Override
public ApplicationMergeEntity newMr(ApplicationMergeEntity entity) throws IOException {
entity.setCreateTimestamp(System.currentTimeMillis());
entity.setCreateUserId(StpUtil.getLoginIdAsString());
// 默认状态
entity.setStatus(MergeRequestStatus.OPEN.toString());
// start_commit_id
String workspaceId = entity.getWorkspaceId();
File gitDir = workspaceService.getGitDir(workspaceId);
try (Repository repository = JGitUtils.openRepository(gitDir)) {
ObjectId branchAId = repository.resolve(entity.getSourceBranch());
ObjectId branchBId = repository.resolve(entity.getTargetBranch());
RevCommit mergeBaseCommit = JGitUtils.getMergeBaseCommit(repository, branchAId, branchBId);
entity.setStartCommitId(mergeBaseCommit.getName());
}
// save
this.save(entity);
SysUserEntity user = userService.getById(entity.getCreateUserId());
entity.setCreateUser(user);
return entity;
}
@Override
public List<Map<Object, Object>> listMrCommit(String mrId) throws IOException {
ApplicationMergeEntity entity = this.getById(mrId);
T.VerifyUtil.is(entity).notEmpty(RCode.SYS_RECORD_NOT_FOUND);
String workspaceId = entity.getWorkspaceId();
File gitDir = workspaceService.getGitDir(workspaceId);
try (Repository repository = JGitUtils.openRepository(gitDir)) {
String oldCommitId = entity.getStartCommitId();
String newCommitId = null;
MergeRequestStatus mergeRequestStatus = MergeRequestStatus.getInstance(entity.getStatus());
switch (mergeRequestStatus) {
case OPEN: {
newCommitId = JGitUtils.getBranchLatestCommit(repository, entity.getSourceBranch()).getName();
break;
}
case CLOSED:
case MERGED: {
newCommitId = entity.getEndCommitId();
break;
}
}
// newCommitId-oldCommitId 期间的提交信息
List<RevCommit> revCommits = JGitUtils.listCommitRange(repository, newCommitId, oldCommitId);
List<Map<Object, Object>> list = T.ListUtil.list(true);
revCommits.forEach(revCommit -> list.add(JGitUtils.buildAswCommitInfo(revCommit)));
return list;
}
}
@Override
public Map<Object, Object> mrCommitDiff(String mrId) {
ApplicationMergeEntity entity = this.getById(mrId);
T.VerifyUtil.is(entity).notEmpty(RCode.SYS_RECORD_NOT_FOUND);
String workspaceId = entity.getWorkspaceId();
File gitDir = workspaceService.getGitDir(workspaceId);
try (Repository repository = JGitUtils.openRepository(gitDir)) {
String sourceBranch = entity.getSourceBranch();
String targetBranch = entity.getTargetBranch();
// 获取 sourceBranch,targetBranch 合并冲突文件
List<String> conflictFileList = T.ListUtil.list(true);
// src branch changed files
String oldCommitId = entity.getStartCommitId();
String newCommitId = null;
MergeRequestStatus mergeRequestStatus = MergeRequestStatus.getInstance(entity.getStatus());
switch (mergeRequestStatus) {
case OPEN: {
RevCommit branchLatestCommit = JGitUtils.getBranchLatestCommit(repository, sourceBranch);
newCommitId = branchLatestCommit.getName();
if (JGitUtils.isBranchExists(repository, targetBranch)) {
// open 状态下,获取 sourceBranch,targetBranch 冲突文件
conflictFileList = JGitUtils.getConflictFilePathInBranches(repository, sourceBranch, targetBranch);
}
break;
}
case CLOSED:
case MERGED: {
newCommitId = entity.getEndCommitId();
break;
}
}
List<Map<Object, Object>> diffFileListInCommits = JGitUtils.getDiffFileListInCommits(repository, newCommitId, oldCommitId);
List<String> finalConflictFileList = conflictFileList;
diffFileListInCommits.parallelStream()
.forEach(m -> {
T.MapUtil.renameKey(m, "oldContent", "sourceContent");
T.MapUtil.renameKey(m, "newContent", "targetContent");
T.MapUtil.renameKey(m, "oldPath", "sourcePath");
T.MapUtil.renameKey(m, "newPath", "targetPath");
String sourcePath = T.MapUtil.getStr(m, "sourcePath");
String targetPath = T.MapUtil.getStr(m, "targetPath");
m.put("conflict", false);
if (finalConflictFileList.contains(sourcePath) || finalConflictFileList.contains(targetPath)) {
m.put("conflict", true);
}
});
RevCommit sourceCommit = JGitUtils.infoCommit(repository, oldCommitId);
RevCommit targetCommit = JGitUtils.infoCommit(repository, newCommitId);
Map<Object, Object> m = T.MapUtil.builder()
.put("sourceBranch", sourceBranch)
.put("targetBranch", targetBranch)
.put("sourceCommitId", oldCommitId)
.put("targetCommitId", newCommitId)
.put("sourceCommit", JGitUtils.buildAswCommitInfo(sourceCommit))
.put("targetCommit", JGitUtils.buildAswCommitInfo(targetCommit))
.put("files", diffFileListInCommits)
.build();
return m;
} catch (IOException e) {
log.error(e, "[mrCommitDiff] [error] [id: {}]", mrId);
throw new RuntimeException(e);
}
}
@Override
public Map<Object, Object> getMrConflictList(String mrId) throws IOException {
ApplicationMergeEntity entity = this.getById(mrId);
T.VerifyUtil.is(entity).notEmpty(RCode.SYS_RECORD_NOT_FOUND);
String workspaceId = entity.getWorkspaceId();
File gitDir = workspaceService.getGitDir(workspaceId);
try (Repository repository = JGitUtils.openRepository(gitDir)) {
String sourceBranch = entity.getSourceBranch();
String targetBranch = entity.getTargetBranch();
List<Map<Object, Object>> conflictFileInfoInBranches = JGitUtils.getConflictFileInfoInBranches(repository, sourceBranch, targetBranch);
if (T.CollUtil.isNotEmpty(conflictFileInfoInBranches)) {
// srcBranch->tgtBranch 有冲突,先从 tgtBranch merge 到 srcBranch在 srcBranch 处理后再合并,参考 gitlab
StringBuilder commitMessage = new StringBuilder();
commitMessage.append("Merge branch '").append(targetBranch).append("' into '").append(sourceBranch).append("'\n\n");
commitMessage.append("# Conflicts:\n");
for (Map<Object, Object> map : conflictFileInfoInBranches) {
String path = T.MapUtil.getStr(map, "path");
commitMessage.append("# ").append(path).append("\n");
}
String parentId = JGitUtils.getBranchLatestCommit(repository, sourceBranch).getName();
Map<Object, Object> m = T.MapUtil.builder()
.put("sourceBranch", sourceBranch)
.put("targetBranch", targetBranch)
.put("commitId", parentId)
.put("commitMessage", commitMessage.toString())
.put("files", conflictFileInfoInBranches)
.build();
return m;
}
return T.MapUtil.builder()
.put("sourceBranch", sourceBranch)
.put("targetBranch", targetBranch)
.put("commitId", null)
.put("commitMessage", null)
.put("files", conflictFileInfoInBranches)
.build();
}
}
/**
* resolve mr conflicts
*
* @param mrId
* @param body
*/
@Override
public void resolveMrConflicts(String mrId, Map<String, Object> body) throws IOException {
ApplicationMergeEntity entity = this.getById(mrId);
T.VerifyUtil.is(entity).notEmpty(RCode.SYS_RECORD_NOT_FOUND);
String workspaceId = entity.getWorkspaceId();
File gitDir = workspaceService.getGitDir(workspaceId);
try (Repository repository = JGitUtils.openRepository(gitDir)) {
SysUserEntity loginUserEntity = userService.getById(StpUtil.getLoginIdAsString());
String author = loginUserEntity.getName();
String sourceBranch = entity.getSourceBranch();
String targetBranch = entity.getTargetBranch();
String commitMessage = T.MapUtil.getStr(body, "commitMessage");
List<Map<String, String>> files = T.MapUtil.get(body, "files", List.class, new ArrayList());
// 反过来合并
JGitUtils.mergeBranch(repository, targetBranch, sourceBranch, author, commitMessage, files);
} catch (GitAPIException e) {
log.error(e, "[resolveMrConflicts] [error] [id: {}]", mrId);
throw new RuntimeException(e);
}
}
/**
* merge operation
*
* @param mrId
* @param action merge|close
* @return
*/
@Override
public ApplicationMergeEntity mergeMr(String mrId, String action) {
ApplicationMergeEntity entity = this.getById(mrId);
T.VerifyUtil.is(entity).notEmpty(RCode.SYS_RECORD_NOT_FOUND);
String workspaceId = entity.getWorkspaceId();
File gitDir = workspaceService.getGitDir(workspaceId);
try (Repository repository = JGitUtils.openRepository(gitDir)) {
switch (action) {
case "merge": {
MergeRequestStatus mergeRequestStatus = MergeRequestStatus.getInstance(entity.getStatus());
if (!mergeRequestStatus.canMerge()) {
throw new ASWException(RCode.GIT_MERGE_NOT_SUPPORTED.setParam(mergeRequestStatus.toString()));
}
String srcBranch = entity.getSourceBranch();
String tgtBranch = entity.getTargetBranch();
StringBuilder commitMessage = new StringBuilder();
commitMessage.append("Merge branch '").append(srcBranch).append("' into '").append(tgtBranch).append("'\n\n");
commitMessage.append(entity.getTitle());
commitMessage.append("\n");
// 检查 tgtBranch 是否存在
if (T.BooleanUtil.negate(JGitUtils.isBranchExists(repository, tgtBranch))) {
throw new ASWException(RCode.GIT_MERGE_TARGET_BRANCH_NOT_EXIST.setParam(tgtBranch));
}
// merge
try {
SysUserEntity loginUserEntity = userService.getById(StpUtil.getLoginIdAsString());
JGitUtils.mergeBranch(repository, srcBranch, tgtBranch, loginUserEntity.getName(), commitMessage.toString(), null);
String latestCommitId = JGitUtils.getBranchLatestCommit(repository, srcBranch).getName();
entity.setEndCommitId(latestCommitId);
entity.setStatus(MergeRequestStatus.MERGED.toString());
} catch (Exception e) {
log.error(e, "[mergeMr] [merge error] [workspaceId: {}] [srcBranch: {}] [tgtBranch: {}] [msg: {}]", workspaceId, srcBranch, tgtBranch, e.getMessage());
throw new ASWException(RCode.GIT_MERGE_FAILED.setParam(e.getMessage()));
}
// remove source branch
if (1 == entity.getRemoveSourceBranch()) {
try {
log.info("[mergeMr] [remove source branch] [workspaceId: {}] [branch: {}]", workspaceId, srcBranch);
branchService.deleteBranch(workspaceId, srcBranch);
} catch (Exception e) {
log.error(e, "[mergeMr] [remove source branch error] [workspaceId: {}] [branch: {}] [msg: {}]", workspaceId, srcBranch, e.getMessage());
}
}
// update
this.update(new LambdaUpdateWrapper<ApplicationMergeEntity>()
.eq(ApplicationMergeEntity::getId, mrId)
.set(ApplicationMergeEntity::getEndCommitId, entity.getEndCommitId())
.set(ApplicationMergeEntity::getStatus, entity.getStatus())
.set(ApplicationMergeEntity::getMessage, entity.getMessage())
);
return entity;
}
case "close": {
String updateStatus = StrUtil.equals(MergeRequestStatus.OPEN.toString(), entity.getStatus()) ? MergeRequestStatus.CLOSED.toString() : entity.getStatus();
entity.setStatus(updateStatus);
// 关闭请求,保留当前 src branch last commit = end_commit_id
String latestCommitId = JGitUtils.getBranchLatestCommit(repository, entity.getSourceBranch()).getName();
this.update(new LambdaUpdateWrapper<ApplicationMergeEntity>()
.eq(ApplicationMergeEntity::getId, mrId)
.set(ApplicationMergeEntity::getStatus, updateStatus)
.set(ApplicationMergeEntity::getEndCommitId, latestCommitId)
);
return entity;
}
default:
break;
}
return entity;
} catch (IOException e) {
log.error(e, "[mergeMr] [error] [id: {}] [action: {}]", mrId, action);
throw new RuntimeException(e);
}
}
public static enum MergeRequestStatus {
OPEN {
// 默认状态
public String toString() {
return "open";
}
public boolean canMerge() {
return true;
}
},
MERGED {
// 合并成功
public String toString() {
return "merged";
}
public boolean canMerge() {
return false;
}
},
CLOSED {
// 合并关闭
public String toString() {
return "closed";
}
public boolean canMerge() {
return false;
}
};
private MergeRequestStatus() {
}
public static MergeRequestStatus getInstance(String name) {
for (MergeRequestStatus v : values()) {
if (StrUtil.equalsIgnoreCase(v.name(), name)) {
return v;
}
}
return null;
}
public abstract boolean canMerge();
}
}

View File

@@ -0,0 +1,184 @@
package net.geedge.asw.module.app.service.impl;
import cn.dev33.satoken.stp.StpUtil;
import cn.hutool.log.Log;
import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
import com.baomidou.mybatisplus.core.conditions.update.LambdaUpdateWrapper;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;
import net.geedge.asw.common.config.Query;
import net.geedge.asw.common.util.*;
import net.geedge.asw.module.app.dao.ApplicationReleaseDao;
import net.geedge.asw.module.app.entity.ApplicationReleaseEntity;
import net.geedge.asw.module.app.service.IApplicationReleaseService;
import net.geedge.asw.module.app.util.JGitUtils;
import net.geedge.asw.module.sys.entity.SysUserEntity;
import net.geedge.asw.module.sys.service.ISysUserService;
import net.geedge.asw.module.workspace.entity.WorkspaceEntity;
import net.geedge.asw.module.workspace.service.IWorkspaceService;
import org.eclipse.jgit.api.errors.GitAPIException;
import org.eclipse.jgit.lib.Ref;
import org.eclipse.jgit.lib.Repository;
import org.eclipse.jgit.revwalk.RevCommit;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
import java.io.File;
import java.io.IOException;
import java.util.List;
import java.util.Map;
@Service
public class ApplicationReleaseServiceImpl extends ServiceImpl<ApplicationReleaseDao, ApplicationReleaseEntity> implements IApplicationReleaseService {
private final static Log log = Log.get();
@Value("${asw.resources.path:resources}")
private String aswResourcesPath;
@Autowired
private ISysUserService userService;
@Autowired
private IWorkspaceService workspaceService;
@Override
public ApplicationReleaseEntity queryInfo(String id) {
ApplicationReleaseEntity entity = this.getById(id);
T.VerifyUtil.is(entity).notNull(RCode.SYS_RECORD_NOT_FOUND);
SysUserEntity createUser = userService.getById(entity.getCreateUserId());
SysUserEntity updateUser = userService.getById(entity.getUpdateUserId());
createUser.setPwd(null);
updateUser.setPwd(null);
entity.setCreateUser(createUser);
entity.setUpdateUser(updateUser);
File gitDir = workspaceService.getGitDir(entity.getWorkspaceId());
try (Repository repository = JGitUtils.openRepository(gitDir);) {
Ref tagRef = repository.findRef(JGitUtils.getFullTagName(entity.getTagName()));
String tgtCommitId = tagRef.getTarget().getObjectId().getName();
RevCommit revCommit = JGitUtils.infoCommit(repository, tgtCommitId);
entity.setCommit(JGitUtils.buildAswCommitInfo(revCommit));
return entity;
} catch (IOException e) {
log.error(e, "[queryInfo] [id: {}]", id);
throw new RuntimeException(e);
}
}
@Override
public Page queryList(Map<String, Object> params) {
Page page = new Query(ApplicationReleaseEntity.class).getPage(params);
List<ApplicationReleaseEntity> entityList = this.getBaseMapper().queryList(page, params);
page.setRecords(entityList);
return page;
}
@Override
public ApplicationReleaseEntity saveRelease(String workspaceId, String name, String tagName, String description) {
ApplicationReleaseEntity checkTagInUse = this.getOne(new LambdaQueryWrapper<ApplicationReleaseEntity>()
.eq(ApplicationReleaseEntity::getWorkspaceId, workspaceId)
.eq(ApplicationReleaseEntity::getTagName, tagName)
);
if (null != checkTagInUse) {
throw new ASWException(RCode.GIT_TAG_ALREADY_IN_USE);
}
String uuid = T.StrUtil.uuid();
// release file
File releaseFile = FileResourceUtil.createFile(this.aswResourcesPath, workspaceId, Constants.FileTypeEnum.RELEASE.getType(), uuid, uuid + ".zip");
File gitDir = workspaceService.getGitDir(workspaceId);
try (Repository repository = JGitUtils.openRepository(gitDir);) {
Ref tagRef = repository.findRef(JGitUtils.getFullTagName(tagName));
T.VerifyUtil.is(tagRef).notNull(RCode.GIT_TAG_NOT_FOUND.setParam(tagName));
// archive
WorkspaceEntity workspace = workspaceService.getById(workspaceId);
String prefix = T.StrUtil.concat(true, workspace.getName(), "-", tagName, "/");
JGitUtils.archive(repository, tagName, releaseFile, prefix, "zip");
ApplicationReleaseEntity entity = new ApplicationReleaseEntity();
entity.setId(uuid);
entity.setName(name);
entity.setTagName(tagName);
entity.setPath(releaseFile.getAbsolutePath());
entity.setDescription(description);
entity.setWorkspaceId(workspaceId);
entity.setCreateUserId(StpUtil.getLoginIdAsString());
entity.setCreateTimestamp(System.currentTimeMillis());
entity.setUpdateUserId(StpUtil.getLoginIdAsString());
entity.setUpdateTimestamp(System.currentTimeMillis());
this.save(entity);
// build record info
SysUserEntity createUser = userService.getById(entity.getCreateUserId());
SysUserEntity updateUser = userService.getById(entity.getUpdateUserId());
createUser.setPwd(null);
updateUser.setPwd(null);
entity.setCreateUser(createUser);
entity.setUpdateUser(updateUser);
String tgtCommitId = tagRef.getTarget().getObjectId().getName();
RevCommit revCommit = JGitUtils.infoCommit(repository, tgtCommitId);
entity.setCommit(JGitUtils.buildAswCommitInfo(revCommit));
return entity;
} catch (IOException | GitAPIException e) {
T.FileUtil.del(releaseFile);
log.error(e, "[saveRelease] [error] [workspaceId: {}] [name: {}] [tagName: {}]", workspaceId, name, tagName);
throw new RuntimeException(e);
}
}
@Override
public ApplicationReleaseEntity updateRelease(String id, String name, String description) {
ApplicationReleaseEntity entity = this.getById(id);
T.VerifyUtil.is(entity).notNull(RCode.SYS_RECORD_NOT_FOUND);
LambdaUpdateWrapper<ApplicationReleaseEntity> wrapper = new LambdaUpdateWrapper<ApplicationReleaseEntity>()
.eq(ApplicationReleaseEntity::getId, id)
.set(ApplicationReleaseEntity::getName, name)
.set(ApplicationReleaseEntity::getDescription, description)
.set(ApplicationReleaseEntity::getUpdateUserId, StpUtil.getLoginIdAsString())
.set(ApplicationReleaseEntity::getUpdateTimestamp, System.currentTimeMillis());
// update
this.update(wrapper);
// return record info
return this.queryInfo(id);
}
@Override
@Transactional(rollbackFor = Exception.class)
public void removeRelease(String id) {
ApplicationReleaseEntity entity = this.getById(id);
if (null != entity) {
// del file
T.FileUtil.del(entity.getPath());
// del record
this.removeById(id);
}
}
@Override
@Transactional(rollbackFor = Exception.class)
public void removeRelease(String workspaceId, String tagName) {
ApplicationReleaseEntity entity = this.getOne(new LambdaQueryWrapper<ApplicationReleaseEntity>()
.eq(ApplicationReleaseEntity::getWorkspaceId, workspaceId)
.eq(ApplicationReleaseEntity::getTagName, tagName)
);
if (null != entity) {
// del file
T.FileUtil.del(entity.getPath());
// del record
this.removeById(entity.getId());
}
}
}

View File

@@ -5,244 +5,767 @@ import cn.hutool.json.JSON;
import cn.hutool.json.JSONConfig; import cn.hutool.json.JSONConfig;
import cn.hutool.json.JSONObject; import cn.hutool.json.JSONObject;
import cn.hutool.log.Log; import cn.hutool.log.Log;
import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;
import net.geedge.asw.common.util.ASWException; import net.geedge.asw.common.util.ASWException;
import net.geedge.asw.common.util.RCode; import net.geedge.asw.common.util.RCode;
import net.geedge.asw.common.util.T; import net.geedge.asw.common.util.T;
import net.geedge.asw.module.app.dao.ApplicationDao; import net.geedge.asw.module.app.entity.ApplicationEntity;
import net.geedge.asw.module.app.entity.*; import net.geedge.asw.module.app.service.IApplicationService;
import net.geedge.asw.module.app.service.*; import net.geedge.asw.module.app.service.ITSGApplicationService;
import net.geedge.asw.module.app.util.JGitUtils;
import net.geedge.asw.module.sys.entity.SysUserEntity; import net.geedge.asw.module.sys.entity.SysUserEntity;
import net.geedge.asw.module.sys.service.ISysUserService; import net.geedge.asw.module.sys.service.ISysUserService;
import net.geedge.asw.module.workspace.service.IWorkspaceService;
import org.eclipse.jgit.api.Git;
import org.eclipse.jgit.api.errors.ConcurrentRefUpdateException;
import org.eclipse.jgit.api.errors.GitAPIException;
import org.eclipse.jgit.diff.DiffEntry;
import org.eclipse.jgit.diff.DiffFormatter;
import org.eclipse.jgit.dircache.DirCache;
import org.eclipse.jgit.dircache.DirCacheBuilder;
import org.eclipse.jgit.dircache.DirCacheEntry;
import org.eclipse.jgit.lib.*;
import org.eclipse.jgit.revwalk.RevCommit;
import org.eclipse.jgit.revwalk.RevTree;
import org.eclipse.jgit.revwalk.RevWalk;
import org.eclipse.jgit.treewalk.CanonicalTreeParser;
import org.eclipse.jgit.treewalk.TreeWalk;
import org.eclipse.jgit.treewalk.filter.PathFilter;
import org.eclipse.jgit.util.io.DisabledOutputStream;
import org.springframework.beans.factory.annotation.Autowired; import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Service; import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
import java.util.ArrayList; import java.io.File;
import java.util.List; import java.io.IOException;
import java.util.Map; import java.nio.file.Path;
import java.util.*;
import java.util.stream.Collectors;
@Service @Service
public class ApplicationServiceImpl extends ServiceImpl<ApplicationDao, ApplicationEntity> implements IApplicationService { public class ApplicationServiceImpl implements IApplicationService {
private static final Log log = Log.get(); private static final Log log = Log.get();
@Autowired @Value("${asw.application.template.meta.json}")
private IApplicationLogService applicationLogService; private String metaJsonTemplate;
@Value("${asw.application.template.signature.json}")
private String signatureJsonTemplate;
@Autowired @Autowired
private ISysUserService userService; private ISysUserService userService;
@Autowired @Autowired
private IApplicationSignatureService signatureService; private IWorkspaceService workspaceService;
@Autowired @Autowired
private IApplicationNoteService noteService; @Qualifier("tsg2402ApplicationService")
private ITSGApplicationService tsg2402ApplicationService;
@Autowired
private IApplicationHrefService hrefService;
@Autowired
private IApplicationAttachmentService attachmentService;
@Autowired
private ITSGApplicationService tsgApplicationService;
@Override @Override
public ApplicationEntity detail(String id, String workspaceId) { public Map<Object, Object> infoApplication(String workspaceId, String branch, String name) {
ApplicationEntity app = this.getOne(new LambdaQueryWrapper<ApplicationEntity>() Map<Object, Object> result = T.MapUtil.builder()
.eq(ApplicationEntity::getId, id) .put("branch", branch)
.eq(ApplicationEntity::getWorkspaceId, workspaceId)); .build();
ApplicationSignatureEntity signature = signatureService.getOne(new LambdaQueryWrapper<ApplicationSignatureEntity>() File gitDir = workspaceService.getGitDir(workspaceId);
.eq(ApplicationSignatureEntity::getApplicationId, app.getId()) try (Repository repository = JGitUtils.openRepository(gitDir)) {
.orderByDesc(ApplicationSignatureEntity::getOpVersion) ObjectId branchRef = repository.resolve(branch);
.last("limit 1")); result.put("commitId", branchRef.getName());
app.setSignature(signature);
ApplicationNoteEntity note = noteService.getOne(new LambdaQueryWrapper<ApplicationNoteEntity>() List<Map<Object, Object>> files = T.ListUtil.list(true);
.eq(ApplicationNoteEntity::getApplicationId, app.getId()) try (TreeWalk treeWalk = new TreeWalk(repository);
.orderByDesc(ApplicationNoteEntity::getOpVersion) RevWalk revWalk = new RevWalk(repository)) {
.last("limit 1"));
app.setNote(note);
List<ApplicationAttachmentEntity> attachmentEntityList = attachmentService.list(new LambdaQueryWrapper<ApplicationAttachmentEntity>() treeWalk.addTree(revWalk.parseTree(branchRef));
.eq(ApplicationAttachmentEntity::getApplicationId, app.getId())); treeWalk.setFilter(PathFilter.create("applications/" + name + "/"));
attachmentEntityList.stream().forEach(x -> x.setPath(null)); treeWalk.setRecursive(true);
app.setAttatchments(attachmentEntityList);
List<ApplicationHrefEntity> hrefEntityList = hrefService.list(new LambdaQueryWrapper<ApplicationHrefEntity>() Map<String, String> appFilePathMapping = T.MapUtil.newHashMap(true);
.eq(ApplicationHrefEntity::getApplicationId, app.getId())); while (treeWalk.next()) {
app.setHrefs(hrefEntityList); String pathString = treeWalk.getPathString();
Map<Object, Object> m = T.MapUtil.builder()
.put("path", pathString)
.build();
SysUserEntity createUser = userService.getById(app.getCreateUserId()); Map<Object, Object> fileContent = JGitUtils.getFileContent(repository, pathString, treeWalk.getObjectId(0));
SysUserEntity updateUser = userService.getById(app.getUpdateUserId()); m.putAll(fileContent);
app.setCreateUser(createUser); files.add(m);
app.setUpdateUser(updateUser);
return app; appFilePathMapping.put(pathString, pathString);
}
Map<String, RevCommit> lastCommitMapping = this.getLastCommitIdDataByPath(repository, branch, appFilePathMapping);
for (Map<Object, Object> m : files) {
String path = T.MapUtil.getStr(m, "path");
RevCommit revCommit = lastCommitMapping.get(path);
m.put("lastCommitId", revCommit != null ? revCommit.getName() : null);
}
}
result.put("files", files);
} catch (IOException e) {
log.error(e, "[infoApplication] [error] [workspaceId: {}] [branch: {}] [name: {}]", workspaceId, branch, name);
throw new RuntimeException(e);
}
return result;
} }
@Override @Override
public Page queryList(Map<String, Object> params) { public List<Map<Object, Object>> listApplication(String workspaceId, String branch, String q) {
Page page = T.PageUtil.getPage(params); List<Map<Object, Object>> resultList = T.ListUtil.list(true);
List<ApplicationEntity> packageList = this.getBaseMapper().queryList(page, params);
page.setRecords(packageList); File gitDir = workspaceService.getGitDir(workspaceId);
return page; try (Repository repository = JGitUtils.openRepository(gitDir);
TreeWalk treeWalk = new TreeWalk(repository);
RevWalk revWalk = new RevWalk(repository);) {
ObjectId branchRef = repository.resolve(branch);
treeWalk.addTree(revWalk.parseTree(branchRef));
treeWalk.setFilter(PathFilter.create("applications/"));
treeWalk.setRecursive(true);
Map<String, String> appDirPathMapping = T.MapUtil.newHashMap(true);
while (treeWalk.next()) {
String filePath = treeWalk.getPathString();
String fileName = treeWalk.getNameString();
if (T.StrUtil.equals("meta.json", fileName)) {
// application_name 从目录中获取
String applicationName = T.PathUtil.getPathEle(Path.of(filePath), 1).toString();
// filter by name
if (T.StrUtil.isNotEmpty(q) && !T.StrUtil.containsIgnoreCase(applicationName, q)) {
continue;
} }
private void validateParam(ApplicationEntity entity, boolean isUpdate) { ObjectLoader loader = repository.open(treeWalk.getObjectId(0));
ApplicationEntity one = this.getOne(new LambdaQueryWrapper<ApplicationEntity>() String metaJsonStr = T.StrUtil.utf8Str(loader.getBytes());
.eq(ApplicationEntity::getWorkspaceId, entity.getWorkspaceId()) metaJsonStr = T.StrUtil.emptyToDefault(metaJsonStr, T.StrUtil.EMPTY_JSON);
.eq(ApplicationEntity::getName, entity.getName())); Map metaJsonMap = T.MapUtil.empty();
try {
if (T.ObjectUtil.isNotNull(one) && !isUpdate || T.ObjectUtil.isNotNull(one) && isUpdate && !T.StrUtil.equals(entity.getId(), one.getId())) { metaJsonMap = T.JSONUtil.toBean(metaJsonStr, Map.class);
throw ASWException.builder().rcode(RCode.APP_DUPLICATE_RECORD).build(); } catch (Exception e) {
log.error(e, "[listApplication] [meat.json format error] [application: {}]", applicationName);
} }
// package name format Map<Object, Object> m = T.MapUtil.newHashMap(true);
if (T.ObjectUtil.isNotEmpty(entity.getPackageName()) && !T.JSONUtil.isTypeJSON(entity.getPackageName())) { m.putAll(metaJsonMap);
throw ASWException.builder().rcode(RCode.APP_PACKAGE_NAME_FORMAT_ERROR).build(); m.put("name", applicationName);
} else if (T.ObjectUtil.isEmpty(entity.getPackageName())) {
entity.setPackageName("{}");
}
// tags name format String appDirPath = treeWalk.getPathString().replaceAll(fileName, "");
if (T.StrUtil.isNotEmpty(entity.getTags()) && !T.JSONUtil.isTypeJSON(entity.getTags())) { appDirPathMapping.put(applicationName, appDirPath);
throw ASWException.builder().rcode(RCode.APP_TAGS_FORMAT_ERROR).build();
}
// signature resultList.add(m);
if (T.ObjectUtil.isNotEmpty(entity.getSignature())) {
if (!T.StrUtil.isNotEmpty(entity.getSignature().getContent())){
throw ASWException.builder().rcode(RCode.APP_SIGNATURE_CONTENT_CANNOT_EMPTY).build();
}
if (!T.JSONUtil.isTypeJSON(entity.getSignature().getContent())){
throw ASWException.builder().rcode(RCode.APP_SIGNATURE_CONTENT_CANNOT_EMPTY).build();
} }
} }
// note Map<String, RevCommit> lastCommitMapping = this.getLastCommitIdDataByPath(repository, branch, appDirPathMapping);
/* if (T.ObjectUtil.isNotEmpty(entity.getNote()) && !T.StrUtil.isNotEmpty(entity.getNote().getContent())) {
throw ASWException.builder().rcode(RCode.APP_NOTE_CONTENT_CANNOT_EMPTY).build();
}*/
// properties for (Map<Object, Object> map : resultList) {
if (T.ObjectUtil.isNotEmpty(entity.getProperties()) && !T.JSONUtil.isTypeJSON(entity.getProperties().toString())) { String applicationName = T.MapUtil.getStr(map, "name");
throw ASWException.builder().rcode(RCode.APP_PROPERTIES_FORMAT_ERROR).build(); RevCommit lastCommit = T.MapUtil.get(lastCommitMapping, applicationName, RevCommit.class);
map.put("commit", JGitUtils.buildAswCommitInfo(lastCommit));
} }
// 按照提交日期倒叙
resultList = resultList.stream().sorted(Comparator.comparing(map -> {
Map commit = T.MapUtil.get((Map) map, "commit", Map.class, new HashMap(2));
return (Long) T.MapUtil.getLong(commit, "createdAt", 0l);
}).reversed()).collect(Collectors.toList());
} catch (IOException e) {
log.error(e, "[listApplication] [error] [workspaceId: {}] [branch: {}]", workspaceId, branch);
throw new RuntimeException(e);
}
return resultList;
} }
@Override @Override
@Transactional(rollbackFor = Exception.class) public List<ApplicationEntity> listApplication(String workspaceId, String branch, String... names) {
public ApplicationEntity saveApplication(ApplicationEntity entity) { List<ApplicationEntity> resultList = T.ListUtil.list(true);
this.validateParam(entity, false); File gitDir = workspaceService.getGitDir(workspaceId);
try (Repository repository = JGitUtils.openRepository(gitDir);
TreeWalk treeWalk = new TreeWalk(repository);
RevWalk revWalk = new RevWalk(repository)) {
ObjectId branchRef = repository.resolve(branch);
treeWalk.addTree(revWalk.parseTree(branchRef));
treeWalk.setFilter(PathFilter.create("applications/"));
treeWalk.setRecursive(true);
Map<String, String> signatureMapping = T.MapUtil.newHashMap();
while (treeWalk.next()) {
String filePath = treeWalk.getPathString();
String applicationName = T.PathUtil.getPathEle(Path.of(filePath), 1).toString();
if (T.BooleanUtil.and(
T.ObjectUtil.isNotEmpty(names),
!Arrays.asList(names).contains(applicationName))) {
continue;
}
switch (treeWalk.getNameString()) {
case "meta.json": {
ObjectLoader loader = repository.open(treeWalk.getObjectId(0));
String metaJsonStr = T.StrUtil.utf8Str(loader.getBytes());
metaJsonStr = T.StrUtil.emptyToDefault(metaJsonStr, T.StrUtil.EMPTY_JSON);
ApplicationEntity application = T.JSONUtil.toBean(metaJsonStr, ApplicationEntity.class);
application.setName(applicationName);
resultList.add(application);
break;
}
case "signature.json": {
ObjectLoader loader = repository.open(treeWalk.getObjectId(0));
signatureMapping.put(applicationName, T.StrUtil.utf8Str(loader.getBytes()));
break;
}
default:
break;
}
}
for (ApplicationEntity entity : resultList) {
String name = entity.getName();
String signature = signatureMapping.getOrDefault(name, "");
if (T.StrUtil.isNotEmpty(signature)) {
JSONObject jsonObject = T.JSONUtil.parseObj(signature);
entity.setSignature(jsonObject);
}
}
} catch (IOException e) {
log.error(e, "[listApplication] [error] [workspaceId: {}] [branch: {}]", workspaceId, branch);
throw new RuntimeException(e);
}
return resultList;
}
@Override
public void newApplication(String workspaceId, String branch, String name) {
File gitDir = workspaceService.getGitDir(workspaceId);
try (Repository repository = JGitUtils.openRepository(gitDir)) {
Map<String, ObjectId> filePathAndBlobIdMap = T.MapUtil.newHashMap(true);
for (String filename : T.ListUtil.of("README.md", "meta.json", "signature.json")) {
String fileContent = T.StrUtil.EMPTY;
if ("meta.json".equals(filename)) {
JSONObject jsonObject = T.JSONUtil.parseObj(this.metaJsonTemplate);
jsonObject.set("id", T.StrUtil.uuid());
jsonObject.set("name", name);
jsonObject.set("longName", name);
fileContent = T.JSONUtil.parse(jsonObject).toJSONString(2);
}
if ("signature.json".equals(filename)) {
fileContent = this.signatureJsonTemplate;
}
ObjectId objectId = JGitUtils.insertBlobFileToDatabase(repository, fileContent.getBytes());
filePathAndBlobIdMap.put(JGitUtils.buildFilePath(name, filename), objectId);
}
try (ObjectInserter inserter = repository.getObjectDatabase().newInserter();
TreeWalk treeWalk = new TreeWalk(repository);
RevWalk revWalk = new RevWalk(repository)) {
// branch tree
ObjectId resolveId = repository.resolve(branch);
if (null != resolveId) {
treeWalk.addTree(revWalk.parseTree(resolveId));
} else {
treeWalk.addTree(new CanonicalTreeParser());
}
treeWalk.setRecursive(true);
DirCache newTree = DirCache.newInCore();
DirCacheBuilder newTreeBuilder = newTree.builder();
while (treeWalk.next()) {
DirCacheEntry entry = JGitUtils.buildDirCacheEntry(treeWalk.getPathString(), treeWalk.getFileMode(0), treeWalk.getObjectId(0));
newTreeBuilder.add(entry);
}
// update new tree
for (Map.Entry<String, ObjectId> entry : filePathAndBlobIdMap.entrySet()) {
String filePath = entry.getKey();
ObjectId blobId = entry.getValue();
// add file ref
DirCacheEntry dirCacheEntry = JGitUtils.buildDirCacheEntry(filePath, FileMode.REGULAR_FILE, blobId);
newTreeBuilder.add(dirCacheEntry);
}
newTreeBuilder.finish();
ObjectId newTreeId = newTree.writeTree(inserter);
String message = String.format("feat: add %s application", name);
SysUserEntity loginUserEntity = userService.getById(StpUtil.getLoginIdAsString());
PersonIdent personIdent = JGitUtils.buildPersonIdent(loginUserEntity.getName());
// commit
JGitUtils.createCommit(repository, branch, newTreeId, message, personIdent);
}
} catch (IOException | ConcurrentRefUpdateException e) {
log.error(e, "[newApplication] [error] [workspaceId: {}] [branch: {}] [name: {}]", workspaceId, branch, name);
throw new RuntimeException(e);
}
}
@Override
public void renameApplication(String workspaceId, String branch, String oldName, String newName) {
log.info("[applicationName] [begin] [workspaceId: {}] [branch: {}] [oldName: {}] [newName: {}]", workspaceId, branch, oldName, newName);
if (T.StrUtil.equals(oldName, newName)) {
log.warn("[renameApplication] [newName has not changed]");
return;
}
File gitDir = workspaceService.getGitDir(workspaceId);
try (Repository repository = JGitUtils.openRepository(gitDir);
TreeWalk treeWalk = new TreeWalk(repository);
RevWalk revWalk = new RevWalk(repository)) {
ObjectId branchRef = repository.resolve(branch);
treeWalk.addTree(revWalk.parseTree(branchRef));
treeWalk.setRecursive(true);
DirCache newTree = DirCache.newInCore();
DirCacheBuilder newTreeBuilder = newTree.builder();
boolean isCommit = false;
String prefix = JGitUtils.buildFilePrefix(oldName);
while (treeWalk.next()) {
String pathString = treeWalk.getPathString();
// rename file
if (pathString.startsWith(prefix)) {
isCommit = true;
String filename = treeWalk.getNameString();
String savePath = JGitUtils.buildFilePath(newName, filename);
if (T.StrUtil.equals("meta.json", filename)) {
// 更新 meta.json name,longName 的值
ObjectLoader loader = repository.open(treeWalk.getObjectId(0));
String metaJsonStr = T.StrUtil.utf8Str(loader.getBytes());
JSONObject jsonObject = T.JSONUtil.parseObj(metaJsonStr);
jsonObject.set("name", newName);
jsonObject.set("longName", newName);
String content = T.JSONUtil.parse(jsonObject).toJSONString(2);
ObjectId blobId = JGitUtils.insertBlobFileToDatabase(repository, content.getBytes());
DirCacheEntry dirCacheEntry = JGitUtils.buildDirCacheEntry(savePath, treeWalk.getFileMode(0), blobId);
newTreeBuilder.add(dirCacheEntry);
} else {
DirCacheEntry dirCacheEntry = JGitUtils.buildDirCacheEntry(savePath, treeWalk.getFileMode(0), treeWalk.getObjectId(0));
newTreeBuilder.add(dirCacheEntry);
}
} else {
// other file
DirCacheEntry entry = JGitUtils.buildDirCacheEntry(pathString, treeWalk.getFileMode(0), treeWalk.getObjectId(0));
newTreeBuilder.add(entry);
}
}
newTreeBuilder.finish();
if (isCommit) {
try (ObjectInserter inserter = repository.getObjectDatabase().newInserter()) {
ObjectId newTreeId = newTree.writeTree(inserter);
String message = String.format("chore: rename application %s to %s", oldName, newName);
SysUserEntity loginUserEntity = userService.getById(StpUtil.getLoginIdAsString());
PersonIdent personIdent = JGitUtils.buildPersonIdent(loginUserEntity.getName());
JGitUtils.createCommit(repository, branch, newTreeId, message, personIdent);
}
} else {
log.warn("[renameApplication] [old application not found] [name: {}]", oldName);
}
} catch (IOException | ConcurrentRefUpdateException e) {
log.error(e, "[renameApplication] [error] [workspaceId: {}] [branch: {}] [oldName: {}] [newName: {}]", workspaceId, branch, oldName, newName);
throw new RuntimeException(e);
}
}
@Override
public void deleteApplication(String workspaceId, String branch, String name) {
File gitDir = workspaceService.getGitDir(workspaceId);
try (Repository repository = JGitUtils.openRepository(gitDir);
ObjectInserter inserter = repository.getObjectDatabase().newInserter();
TreeWalk treeWalk = new TreeWalk(repository);
RevWalk revWalk = new RevWalk(repository);) {
ObjectId branchRef = repository.resolve(branch);
treeWalk.addTree(revWalk.parseTree(branchRef));
treeWalk.setRecursive(true);
DirCache newTree = DirCache.newInCore();
DirCacheBuilder newTreeBuilder = newTree.builder();
String appFilePrefixStr = JGitUtils.buildFilePrefix(name);
while (treeWalk.next()) {
String pathString = treeWalk.getPathString();
if (!pathString.startsWith(appFilePrefixStr)) {
DirCacheEntry entry = JGitUtils.buildDirCacheEntry(pathString, treeWalk.getFileMode(0), treeWalk.getObjectId(0));
newTreeBuilder.add(entry);
}
}
newTreeBuilder.finish();
ObjectId newTreeId = newTree.writeTree(inserter);
String message = String.format("refactor: remove %s application", name);
SysUserEntity loginUserEntity = userService.getById(StpUtil.getLoginIdAsString());
PersonIdent personIdent = JGitUtils.buildPersonIdent(loginUserEntity.getName());
// commit
JGitUtils.createCommit(repository, branch, newTreeId, message, personIdent);
} catch (IOException | ConcurrentRefUpdateException e) {
log.error(e, "[deleteApplication] [error] [workspaceId: {}] [branch: {}] [name: {}]", workspaceId, branch, name);
throw new RuntimeException(e);
}
}
@Override
public void updateApplication(String workspaceId, String branch, String lastCommitId, String message, List<Map<String, String>> updateContent) {
File gitDir = workspaceService.getGitDir(workspaceId);
try (Repository repository = JGitUtils.openRepository(gitDir);
Git git = Git.open(repository.getDirectory());) {
// 分支最新 commitId
ObjectId branchRef = repository.resolve(branch);
// 如果不相等,检查 param.lastCommitId --- branch.HEAD 期间是否更新了本次修改文件
if (!T.StrUtil.equals(lastCommitId, branchRef.getName())) {
List<String> updateFilePath = updateContent.stream()
.filter(map -> {
String action = T.MapUtil.getStr(map, "action");
return T.StrUtil.equalsAny(action, "create", "update");
})
.map(map -> T.MapUtil.getStr(map, "path"))
.collect(Collectors.toList());
ObjectId currentCommitId = repository.resolve(lastCommitId);
try (RevWalk walk = new RevWalk(repository)) {
CanonicalTreeParser c1 = new CanonicalTreeParser();
c1.reset(repository.newObjectReader(), walk.parseCommit(currentCommitId).getTree());
CanonicalTreeParser c2 = new CanonicalTreeParser();
c2.reset(repository.newObjectReader(), walk.parseCommit(branchRef).getTree());
List<DiffEntry> diffs = git.diff()
.setOldTree(c1)
.setNewTree(c2)
.call();
List<String> affectFilePathList = diffs.stream()
.filter(entry -> T.StrUtil.equalsAnyIgnoreCase(entry.getChangeType().name(), DiffEntry.ChangeType.MODIFY.name(), DiffEntry.ChangeType.ADD.name()))
.map(DiffEntry::getNewPath)
.collect(Collectors.toList());
List<String> intersection = T.CollUtil.intersection(updateFilePath, affectFilePathList).stream().toList();
if (T.CollUtil.isEmpty(intersection)) {
// 在 param.lastCommitId 不是当前分支最新提交时,本次提交文件没有冲突,更新 commit=branch.HEAD
lastCommitId = branchRef.getName();
} else {
throw new ASWException(RCode.GIT_COMMIT_CONFLICT_ERROR);
}
}
}
try (ObjectInserter inserter = repository.getObjectDatabase().newInserter();
TreeWalk treeWalk = new TreeWalk(repository);
RevWalk revWalk = new RevWalk(repository)) {
// branch tree
treeWalk.addTree(revWalk.parseTree(branchRef));
treeWalk.setRecursive(true);
DirCache newTree = DirCache.newInCore();
DirCacheBuilder newTreeBuilder = newTree.builder();
List<String> updateFilePath = updateContent.stream().map(map -> T.MapUtil.getStr(map, "path")).collect(Collectors.toList());
while (treeWalk.next()) {
String pathString = treeWalk.getPathString();
// 先删
if (!updateFilePath.contains(pathString)) {
DirCacheEntry entry = JGitUtils.buildDirCacheEntry(pathString, treeWalk.getFileMode(0), treeWalk.getObjectId(0));
newTreeBuilder.add(entry);
}
}
// 后增
for (Map<String, String> map : updateContent) {
String action = T.MapUtil.getStr(map, "action");
if (T.StrUtil.equalsAnyIgnoreCase(action, "create", "update")) {
String path = T.MapUtil.getStr(map, "path");
DirCacheEntry dirCacheEntry = new DirCacheEntry(path);
dirCacheEntry.setFileMode(FileMode.REGULAR_FILE);
String content = T.MapUtil.getStr(map, "content");
String encoding = T.MapUtil.getStr(map, "encoding");
if ("base64".equals(encoding)) {
// binary
byte[] data = Base64.getDecoder().decode(content);
ObjectId objectId = JGitUtils.insertBlobFileToDatabase(repository, data);
dirCacheEntry.setObjectId(objectId);
} else {
// text
ObjectId objectId = JGitUtils.insertBlobFileToDatabase(repository, content.getBytes());
dirCacheEntry.setObjectId(objectId);
}
newTreeBuilder.add(dirCacheEntry);
}
}
newTreeBuilder.finish();
ObjectId newTreeId = newTree.writeTree(inserter);
SysUserEntity loginUserEntity = userService.getById(StpUtil.getLoginIdAsString());
PersonIdent personIdent = JGitUtils.buildPersonIdent(loginUserEntity.getName());
// commit
JGitUtils.createCommit(repository, branch, newTreeId, message, personIdent);
}
} catch (IOException | GitAPIException e) {
log.error(e, "[updateApplication] [error] [workspaceId: {}] [branch: {}] [lastCommitId: {}]", workspaceId, branch, lastCommitId);
throw new RuntimeException(e);
}
}
@Override
public List<Map<Object, Object>> listApplicationCommit(String workspaceId, String branch, String name, String file) {
List<Map<Object, Object>> resultList = T.ListUtil.list(true);
File gitDir = workspaceService.getGitDir(workspaceId);
try (Repository repository = JGitUtils.openRepository(gitDir);
Git git = Git.open(repository.getDirectory());) {
String filterPath = T.StrUtil.concat(true, "applications/", name);
if (T.StrUtil.isNotEmpty(file)) {
filterPath = T.StrUtil.concat(true, filterPath, "/", file);
}
ObjectId branchRef = git.getRepository().resolve(branch);
Iterable<RevCommit> iterable = git.log()
.add(branchRef)
.addPath(filterPath)
.call();
for (RevCommit commit : iterable) {
resultList.add(JGitUtils.buildAswCommitInfo(commit));
}
} catch (IOException | GitAPIException e) {
log.error(e, "[listApplicationCommit] [error] [workspaceId: {}] [branch: {}] [name: {}]", workspaceId, branch, name);
throw new RuntimeException(e);
}
return resultList;
}
@Override
public Map<Object, Object> infoApplicationFileContent(String workspaceId, String branch, String name, String commitId, String file) {
// applications/qq/meta.json
String path = JGitUtils.buildFilePath(name, file);
Map<Object, Object> result = T.MapUtil.builder()
.put("path", path)
.build();
File gitDir = workspaceService.getGitDir(workspaceId);
try (Repository repository = JGitUtils.openRepository(gitDir);
TreeWalk treeWalk = new TreeWalk(repository);
RevWalk revWalk = new RevWalk(repository);
) {
RevCommit revCommit = revWalk.parseCommit(ObjectId.fromString(commitId));
ObjectId treeId = revCommit.getTree().getId();
treeWalk.addTree(treeId);
treeWalk.setRecursive(true);
while (treeWalk.next()) {
if (T.StrUtil.equals(path, treeWalk.getPathString())) {
Map<Object, Object> fileContent = JGitUtils.getFileContent(repository, treeWalk.getPathString(), treeWalk.getObjectId(0));
result.putAll(fileContent);
}
}
} catch (IOException e) {
log.error(e, "[infoApplicationFileContent] [error] [workspaceId: {}] [branch: {}] [name: {}]", workspaceId, branch, name);
throw new RuntimeException(e);
}
return result;
}
/**
* 通过 path 获取 lastCommit
*
* @param repository
* @param branch
* @param pathMapping
* @return
*/
public Map<String, RevCommit> getLastCommitIdDataByPath(Repository repository, String branch, Map<String, String> pathMapping) {
Map<String, RevCommit> result = new HashMap<>();
try (Git git = new Git(repository);
DiffFormatter diffFormatter = new DiffFormatter(DisabledOutputStream.INSTANCE);
) {
diffFormatter.setRepository(repository);
// 指定分支获取 log
Iterable<RevCommit> commits = git.log()
.add(git.getRepository().resolve(branch))
.call();
for (RevCommit commit : commits) {
// 获取上次提交,用以对比差异
RevCommit parent = commit.getParentCount() > 0 ? commit.getParent(0) : null;
if (parent == null) {
continue;
}
// 计算当前提交与上次提交之间的差异
List<DiffEntry> diffs = diffFormatter.scan(parent.getTree(), commit.getTree());
for (DiffEntry diff : diffs) {
String newPath = diff.getNewPath();
String oldPath = diff.getOldPath();
// 检查是否匹配目标路径
for (Map.Entry<String, String> entry : pathMapping.entrySet()) {
String key = entry.getKey();
String path = entry.getValue();
if (T.BooleanUtil.and(
(newPath.startsWith(path) || oldPath.startsWith(path)),
!result.containsKey(key)
)) {
result.put(key, commit);
}
}
}
// 如果所有路径都找到对应的最后提交,提前结束循环
if (result.size() == pathMapping.size()) {
break;
}
}
} catch (IOException | GitAPIException e) {
log.error(e, "[getLastCommitIdDataByPath] [workspace: {}] [branch: {}]", repository.getDirectory().getPath(), branch);
}
return result;
}
public void saveOrUpdateBatch(String workspaceId, String branch, String commitMessage, List<ApplicationEntity> list) {
File gitDir = workspaceService.getGitDir(workspaceId);
try (Repository repository = JGitUtils.openRepository(gitDir);
RevWalk revWalk = new RevWalk(repository)) {
ObjectId branchRef = repository.resolve(branch);
RevTree revTree = revWalk.parseTree(branchRef);
// 先查询,区分 新增APP、修改APP
List<ApplicationEntity> addAppList = T.ListUtil.list(true);
List<ApplicationEntity> updateAppList = T.ListUtil.list(true);
HashMap<String, JSONObject> appMetaContentInDb = T.MapUtil.newHashMap();
try (TreeWalk treeWalk = new TreeWalk(repository)) {
treeWalk.addTree(revTree);
treeWalk.setRecursive(true);
treeWalk.setFilter(PathFilter.create("applications/"));
HashSet<String> appNameListInDb = new HashSet<>();
while (treeWalk.next()) {
String appName = T.PathUtil.getPathEle(Path.of(treeWalk.getPathString()), 1).toString();
appNameListInDb.add(appName);
if (T.StrUtil.equals("meta.json", treeWalk.getNameString())) {
ObjectLoader loader = repository.open(treeWalk.getObjectId(0));
JSONObject jsonObject = T.JSONUtil.parseObj(T.StrUtil.utf8Str(loader.getBytes()));
appMetaContentInDb.put(appName, jsonObject);
}
}
list.parallelStream().forEach(entity -> {
if (appNameListInDb.contains(entity.getName())) {
updateAppList.add(entity);
} else {
addAppList.add(entity);
}
});
}
// 修改文件路径信息
List<String> updateFilePath = T.ListUtil.list(true);
updateAppList.parallelStream().forEach(entity -> {
updateFilePath.add(JGitUtils.buildFilePath(entity.getName(), "meta.json"));
updateFilePath.add(JGitUtils.buildFilePath(entity.getName(), "signature.json"));
});
// build tree
DirCache newTree = DirCache.newInCore();
DirCacheBuilder newTreeBuilder = newTree.builder();
try (TreeWalk treeWalk = new TreeWalk(repository)) {
treeWalk.addTree(revTree);
treeWalk.setRecursive(true);
while (treeWalk.next()) {
// 先删
String filePath = treeWalk.getPathString();
if (!updateFilePath.contains(filePath)) {
DirCacheEntry entry = new DirCacheEntry(filePath);
entry.setFileMode(treeWalk.getFileMode(0));
entry.setObjectId(treeWalk.getObjectId(0));
newTreeBuilder.add(entry);
}
}
// 新增APP
for (ApplicationEntity entity : addAppList) {
for (String fileName : T.ListUtil.of("README.md", "meta.json", "signature.json")) {
String fileContent = T.StrUtil.EMPTY;
if ("meta.json".equals(fileName)) {
JSONObject tempJSONObject = T.JSONUtil.parseObj(entity);
tempJSONObject.remove("signature");
fileContent = T.JSONUtil.parse(tempJSONObject).toJSONString(2);
}
if ("signature.json".equals(fileName)) {
String jsonStr = T.JSONUtil.toJsonStr(entity.getSignature());
JSONObject tempJSONObject = T.JSONUtil.parseObj(jsonStr);
fileContent = T.JSONUtil.parse(tempJSONObject).toJSONString(2);
}
// save // save
entity.setCreateTimestamp(System.currentTimeMillis()); String filePath = JGitUtils.buildFilePath(entity.getName(), fileName);
entity.setUpdateTimestamp(System.currentTimeMillis()); DirCacheEntry dirCacheEntry = new DirCacheEntry(filePath);
entity.setCreateUserId(StpUtil.getLoginIdAsString()); dirCacheEntry.setFileMode(FileMode.REGULAR_FILE);
entity.setUpdateUserId(StpUtil.getLoginIdAsString());
this.save(entity);
if (T.ObjectUtil.isNotEmpty(entity.getSignature())){ ObjectId blobId = JGitUtils.insertBlobFileToDatabase(repository, fileContent.getBytes());
// save signature dirCacheEntry.setObjectId(blobId);
signatureService.saveSignature(entity.getSignature(), entity.getId()); newTreeBuilder.add(dirCacheEntry);
}
}
// 修改APP
for (ApplicationEntity entity : updateAppList) {
// meta.json
JSONObject jsonObject = T.JSONUtil.parseObj(entity);
jsonObject.remove("signature");
JSONObject metaInDb = appMetaContentInDb.get(entity.getName());
if (null != metaInDb) {
// 还原 asw 部分属性值,此部分 tsg app 不记录
jsonObject.set("id", metaInDb.getStr("id"));
jsonObject.set("developer", metaInDb.getStr("developer", ""));
jsonObject.set("website", metaInDb.getStr("website", ""));
jsonObject.set("packageName", metaInDb.getObj("packageName"));
} }
if (T.ObjectUtil.isNotEmpty(entity.getNote())){ String fileContent = T.JSONUtil.parse(jsonObject).toJSONString(2);
//save note ObjectId objectId = JGitUtils.insertBlobFileToDatabase(repository, fileContent.getBytes());
noteService.saveNote(entity.getNote(), entity.getId()); DirCacheEntry dirCacheEntry = JGitUtils.buildDirCacheEntry(JGitUtils.buildFilePath(entity.getName(), "meta.json"), FileMode.REGULAR_FILE, objectId);
newTreeBuilder.add(dirCacheEntry);
// signature.json
String jsonStr = T.JSONUtil.toJsonStr(entity.getSignature());
JSONObject jsonObject2 = T.JSONUtil.parseObj(jsonStr);
String fileContent2 = T.JSONUtil.parse(jsonObject2).toJSONString(2);
ObjectId objectId2 = JGitUtils.insertBlobFileToDatabase(repository, fileContent2.getBytes());
DirCacheEntry dirCacheEntry2 = JGitUtils.buildDirCacheEntry(JGitUtils.buildFilePath(entity.getName(), "signature.json"), FileMode.REGULAR_FILE, objectId2);
newTreeBuilder.add(dirCacheEntry2);
} }
newTreeBuilder.finish();
RevCommit commitId = revWalk.parseCommit(branchRef);
return entity; SysUserEntity loginUserEntity = userService.getById(StpUtil.getLoginIdAsString());
PersonIdent personIdent = JGitUtils.buildPersonIdent(loginUserEntity.getName());
boolean success = JGitUtils.commitIndex(repository, branch, newTree, commitId, null, commitMessage, personIdent);
log.info("[saveOrUpdateBatch] [workspaceId: {}] [branch: {}] [commitId: {}] [success: {}]", workspaceId, branch, commitId, success);
} }
} catch (IOException | ConcurrentRefUpdateException e) {
@Override log.error(e, "[saveOrUpdateBatch] [error] [workspaceId: {}] [branch: {}]", workspaceId, branch);
@Transactional(rollbackFor = Exception.class) throw new RuntimeException(e);
public ApplicationEntity updateApplication(ApplicationEntity entity) {
this.validateParam(entity, true);
ApplicationEntity one = this.getById(entity.getId());
entity.setUpdateTimestamp(System.currentTimeMillis());
entity.setUpdateUserId(StpUtil.getLoginIdAsString());
entity.setOpVersion(one.getOpVersion() + 1);
// update
this.updateById(entity);
// save log
this.saveApplicationToLog(one);
if (T.ObjectUtil.isNotEmpty(entity.getSignature())){
// save signature
signatureService.saveSignature(entity.getSignature(), entity.getId());
} }
if (T.ObjectUtil.isNotEmpty(entity.getNote())){
//save note
noteService.saveNote(entity.getNote(), entity.getId());
}
return entity;
}
private void saveApplicationToLog(ApplicationEntity one) {
ApplicationLogEntity applicationLogEntity = T.BeanUtil.toBean(one, ApplicationLogEntity.class);
applicationLogEntity.setUpdateTimestamp(System.currentTimeMillis());
applicationLogEntity.setUpdateUserId(StpUtil.getLoginIdAsString());
applicationLogEntity.setCreateTimestamp(System.currentTimeMillis());
applicationLogEntity.setCreateUserId(StpUtil.getLoginIdAsString());
applicationLogService.save(applicationLogEntity);
}
@Override
@Transactional(rollbackFor = Exception.class)
public void removeApplication(List<String> ids) {
// remove
this.removeBatchByIds(ids);
applicationLogService.removeBatchByIds(ids);
signatureService.remove(new LambdaQueryWrapper<ApplicationSignatureEntity>().in(ApplicationSignatureEntity::getApplicationId, ids));
noteService.remove(new LambdaQueryWrapper<ApplicationNoteEntity>().in(ApplicationNoteEntity::getApplicationId, ids));
attachmentService.remove(new LambdaQueryWrapper<ApplicationAttachmentEntity>().in(ApplicationAttachmentEntity::getApplicationId, ids));
hrefService.remove(new LambdaQueryWrapper<ApplicationHrefEntity>().in(ApplicationHrefEntity::getApplicationId, ids));
}
@Override
public ApplicationEntity updateBasic(ApplicationEntity entity) {
ApplicationEntity one = this.getById(entity.getId());
if (T.ObjectUtil.isNotNull(one) && !T.StrUtil.equals(entity.getId(), one.getId())) {
throw ASWException.builder().rcode(RCode.APP_DUPLICATE_RECORD).build();
}
// package name format
if (T.ObjectUtil.isNotEmpty(entity.getPackageName()) && !T.JSONUtil.isTypeJSON(entity.getPackageName())) {
throw ASWException.builder().rcode(RCode.APP_PACKAGE_NAME_FORMAT_ERROR).build();
} else if (T.ObjectUtil.isEmpty(entity.getPackageName())) {
entity.setPackageName("{}");
}
// properties
if (T.ObjectUtil.isNotEmpty(entity.getProperties()) && !T.JSONUtil.isTypeJSON(entity.getProperties().toString())) {
throw ASWException.builder().rcode(RCode.APP_PROPERTIES_FORMAT_ERROR).build();
}
entity.setUpdateTimestamp(System.currentTimeMillis());
entity.setUpdateUserId(StpUtil.getLoginIdAsString());
entity.setOpVersion(one.getOpVersion() + 1);
this.saveApplicationToLog(one);
this.updateById(entity);
return entity;
} }
@Override @Override
@@ -250,7 +773,7 @@ public class ApplicationServiceImpl extends ServiceImpl<ApplicationDao, Applicat
try { try {
switch (format) { switch (format) {
case "tsg2402": { case "tsg2402": {
Map<Object, Object> m = tsgApplicationService.aswToTsg2402(appList); Map<Object, Object> m = tsg2402ApplicationService.aswToTsg2402(appList);
JSON json = new JSONObject(m, JSONConfig.create().setIgnoreNullValue(false).setKeyComparator(String::compareToIgnoreCase)); JSON json = new JSONObject(m, JSONConfig.create().setIgnoreNullValue(false).setKeyComparator(String::compareToIgnoreCase));
return T.StrUtil.bytes(json.toJSONString(0)); return T.StrUtil.bytes(json.toJSONString(0));
} }
@@ -265,12 +788,15 @@ public class ApplicationServiceImpl extends ServiceImpl<ApplicationDao, Applicat
} }
@Override @Override
@Transactional(rollbackFor = Exception.class) public List<ApplicationEntity> importAppByFormat(String workspaceId, String branch, String format, List<JSONObject> dataList) {
public List<ApplicationEntity> importAppByFormat(String workspaceId, String format, List<JSONObject> dataList) {
try { try {
switch (format) { switch (format) {
case "tsg2402": { case "tsg2402": {
List<ApplicationEntity> records = tsgApplicationService.tsg2402ToAsw(workspaceId, dataList); List<ApplicationEntity> records = tsg2402ApplicationService.tsg2402ToAsw(workspaceId, dataList);
// commit
String commitMessage = "feat: import application data from TSG system (version 2402)";
this.saveOrUpdateBatch(workspaceId, branch, commitMessage, records);
return records; return records;
} }
default: default:

View File

@@ -0,0 +1,156 @@
package net.geedge.asw.module.app.service.impl;
import cn.hutool.log.Log;
import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
import net.geedge.asw.common.util.ASWException;
import net.geedge.asw.common.util.RCode;
import net.geedge.asw.common.util.T;
import net.geedge.asw.module.app.entity.ApplicationMergeEntity;
import net.geedge.asw.module.app.service.IBranchService;
import net.geedge.asw.module.app.service.IApplicationMergeService;
import net.geedge.asw.module.app.service.impl.ApplicationMergeServiceImpl.MergeRequestStatus;
import net.geedge.asw.module.app.util.JGitUtils;
import net.geedge.asw.module.workspace.service.IWorkspaceService;
import org.eclipse.jgit.api.Git;
import org.eclipse.jgit.api.errors.GitAPIException;
import org.eclipse.jgit.lib.Ref;
import org.eclipse.jgit.lib.Repository;
import org.eclipse.jgit.revwalk.RevCommit;
import org.eclipse.jgit.revwalk.RevWalk;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import java.io.File;
import java.io.IOException;
import java.util.List;
import java.util.Map;
@Service
public class BranchServiceImpl implements IBranchService {
private final static Log log = Log.get();
@Autowired
private IWorkspaceService workspaceService;
@Autowired
private IApplicationMergeService applicationMergeService;
@Override
public List<Map<Object, Object>> listBranch(String workspaceId, String search) {
List<Map<Object, Object>> resultList = T.ListUtil.list(true);
File gitDir = workspaceService.getGitDir(workspaceId);
try (Repository repository = JGitUtils.openRepository(gitDir);
Git git = Git.open(repository.getDirectory())) {
String fullBranch = repository.getFullBranch();
String defaultBranch = "main";
if (fullBranch != null && fullBranch.startsWith(JGitUtils.R_HEADS)) {
defaultBranch = fullBranch.substring(JGitUtils.R_HEADS.length());
}
// 默认行为,进查询本地分支
List<Ref> call = git.branchList().call();
RevWalk revCommits = new RevWalk(repository);
for (Ref ref : call) {
String branchName = ref.getName();
// 返回时去掉前缀
branchName = branchName.replaceAll(JGitUtils.R_HEADS, "");
if (T.StrUtil.isNotEmpty(search)) {
if (!T.StrUtil.contains(branchName, search)) {
continue;
}
}
Map<Object, Object> m = T.MapUtil.builder()
.put("name", branchName)
.put("default", T.StrUtil.equals(defaultBranch, branchName))
.build();
RevCommit commit = revCommits.parseCommit(ref.getObjectId());
m.put("commit", JGitUtils.buildAswCommitInfo(commit));
resultList.add(m);
}
revCommits.close();
revCommits.dispose();
} catch (GitAPIException | IOException e) {
log.error(e, "[listBranch] [error] [workspaceId: {}]", workspaceId);
throw new RuntimeException(e);
}
return resultList;
}
@Override
public Map<Object, Object> infoBranch(String workspaceId, String branchName) {
List<Map<Object, Object>> listBranch = this.listBranch(workspaceId, branchName);
// 分支不存在
if (T.CollUtil.isEmpty(listBranch)) {
throw new ASWException(RCode.SYS_RECORD_NOT_FOUND);
}
return T.CollUtil.getFirst(listBranch);
}
@Override
public Map<Object, Object> newBranch(String workspaceId, String name, String ref) {
File gitDir = workspaceService.getGitDir(workspaceId);
try (Repository repository = JGitUtils.openRepository(gitDir);
Git git = Git.open(repository.getDirectory())) {
git.branchCreate()
.setName(name)
.setStartPoint(ref)
.call();
return this.infoBranch(workspaceId, name);
} catch (GitAPIException | IOException e) {
log.error(e, "[newBranch] [error] [workspaceId: {}] [branch: {}] [ref: {}]", workspaceId, name, ref);
throw new RuntimeException(e);
}
}
@Override
public void deleteBranch(String workspaceId, String branchName) {
log.info("[deleteBranch] [begin] [workspaceId: {}] [branch: {}]", workspaceId, branchName);
File gitDir = workspaceService.getGitDir(workspaceId);
try (Repository repository = JGitUtils.openRepository(gitDir);
Git git = Git.open(repository.getDirectory())) {
git.branchDelete()
.setBranchNames(branchName)
.setForce(true)
.call();
// OPEN 状态mr 源分支被删除mr 记录直接删掉
applicationMergeService.remove(new LambdaQueryWrapper<ApplicationMergeEntity>()
.eq(ApplicationMergeEntity::getWorkspaceId, workspaceId)
.eq(ApplicationMergeEntity::getSourceBranch, branchName)
.eq(ApplicationMergeEntity::getStatus, MergeRequestStatus.OPEN.toString())
);
} catch (GitAPIException | IOException e) {
log.error(e, "[deleteBranch] [error] [workspaceId: {}] [branchName: {}]", workspaceId, branchName);
throw new RuntimeException(e);
}
}
@Override
public List<Map<Object, Object>> listBranchCommit(String workspaceId, String branch, String path) {
List<Map<Object, Object>> resultList = T.ListUtil.list(true);
File gitDir = workspaceService.getGitDir(workspaceId);
try (Repository repository = JGitUtils.openRepository(gitDir)) {
List<RevCommit> branchCommitList = JGitUtils.getBranchCommitList(repository, branch, path);
branchCommitList.forEach(revCommit -> resultList.add(JGitUtils.buildAswCommitInfo(revCommit)));
} catch (GitAPIException | IOException e) {
log.error(e, "[listBranchCommit] [error] [workspaceId: {}] [branch: {}] [path: {}]", workspaceId, branch, path);
throw new RuntimeException(e);
}
return resultList;
}
}

View File

@@ -1,182 +0,0 @@
package net.geedge.asw.module.app.service.impl;
import cn.hutool.log.Log;
import net.geedge.asw.common.util.ASWException;
import net.geedge.asw.common.util.RCode;
import net.geedge.asw.common.util.T;
import net.geedge.asw.module.app.service.IGitService;
import net.geedge.asw.module.workspace.entity.WorkspaceEntity;
import net.geedge.asw.module.workspace.service.IWorkspaceService;
import org.eclipse.jgit.api.Git;
import org.eclipse.jgit.api.errors.GitAPIException;
import org.eclipse.jgit.lib.Config;
import org.eclipse.jgit.lib.PersonIdent;
import org.eclipse.jgit.lib.Ref;
import org.eclipse.jgit.lib.StoredConfig;
import org.eclipse.jgit.revwalk.RevCommit;
import org.eclipse.jgit.revwalk.RevWalk;
import org.eclipse.jgit.storage.file.FileRepositoryBuilder;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import java.io.File;
import java.io.IOException;
import java.util.Arrays;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
@Service
public class GitServiceImpl implements IGitService {
private final static Log log = Log.get();
/**
* 本地分支引用前缀
*/
public static final String LOCAL_BRANCH_PREFIX = "refs/heads/";
@Autowired
private IWorkspaceService workspaceService;
/**
* get repository path
* path= {webRootPath}/workspeace/{workspace.name}
*/
private File getRepoDirPath(String workspaceId) {
WorkspaceEntity workspace = workspaceService.getById(workspaceId);
File repoDir = T.FileUtil.file(T.WebPathUtil.getRootPath(), "workspeace", workspace.getName());
return repoDir;
}
/**
* get git instance
*/
private Git getGitInstance(File repoDir) {
try {
if (T.FileUtil.exist(repoDir) && T.FileUtil.file(repoDir, ".git").exists()) {
log.info("[getGitInstance] [open exist repository] [path: {}]", repoDir);
FileRepositoryBuilder builder = new FileRepositoryBuilder();
builder.setGitDir(T.FileUtil.file(repoDir, ".git"));
builder.readEnvironment();
builder.findGitDir();
return new Git(builder.build());
} else {
log.info("[getGitInstance] [init new repository] [path: {}]", repoDir);
// init
Git git = Git.init().setDirectory(repoDir).call();
// config
StoredConfig config = git.getRepository().getConfig();
config.setString("user", null, "name", "asw");
config.setString("user", null, "email", "asw@geedgenetworks.com");
config.save();
return git;
}
} catch (IOException | GitAPIException | IllegalStateException e) {
log.error(e, "[getGitInstance] [error] [path: {}]", repoDir);
throw new RuntimeException(e);
}
}
@Override
public List<Map<Object, Object>> listBranch(String workspaceId, String search) {
List<Map<Object, Object>> resultList = T.ListUtil.list(true);
File repoDir = this.getRepoDirPath(workspaceId);
try (Git git = this.getGitInstance(repoDir)) {
// List<Ref> call = git.branchList().setListMode(ListBranchCommand.ListMode.ALL).call();
// 默认行为,进查询本地分支
List<Ref> call = git.branchList().call();
RevWalk revCommits = new RevWalk(git.getRepository());
for (Ref ref : call) {
String branchName = ref.getName();
// 返回时去掉前缀
branchName = branchName.replaceAll(LOCAL_BRANCH_PREFIX, "");
if (T.StrUtil.isNotEmpty(search)) {
if (!T.StrUtil.contains(branchName, search)) {
continue;
}
}
Map<Object, Object> m = T.MapUtil.builder()
.put("name", branchName)
.build();
RevCommit commit = revCommits.parseCommit(ref.getObjectId());
List<String> parentIds = Arrays.stream(commit.getParents()).map(RevCommit::getName).collect(Collectors.toList());
Map<Object, Object> m1 = new LinkedHashMap<>();
m1.put("id", commit.getName());
m1.put("shortId", T.StrUtil.subPre(commit.getName(), 8));
m1.put("createdAt", commit.getCommitTime());
m1.put("title", commit.getShortMessage());
m1.put("message", commit.getFullMessage());
m1.put("parentIds", parentIds);
PersonIdent authorIdent = commit.getAuthorIdent();
m1.put("authorName", authorIdent.getName());
m1.put("authorEmail", authorIdent.getEmailAddress());
m1.put("authoredDate", authorIdent.getWhen().getTime());
PersonIdent committerIdent = commit.getCommitterIdent();
m1.put("committerName", committerIdent.getName());
m1.put("committerEmail", committerIdent.getEmailAddress());
m1.put("committedDate", committerIdent.getWhen().getTime());
m.put("commit", m1);
resultList.add(m);
}
} catch (GitAPIException | IOException e) {
log.error(e, "[listBranch] [error] [workspaceId: {}]", workspaceId);
throw new ASWException(RCode.ERROR);
}
return resultList;
}
@Override
public Map<Object, Object> infoBranch(String workspaceId, String branchName) {
List<Map<Object, Object>> listBranch = this.listBranch(workspaceId, branchName);
// 分支不存在
if (T.CollUtil.isEmpty(listBranch)) {
throw new ASWException(RCode.SYS_RECORD_NOT_FOUND);
}
return T.CollUtil.getFirst(listBranch);
}
@Override
public Map<Object, Object> newBranch(String workspaceId, String branchName, String ref) {
File repoDir = this.getRepoDirPath(workspaceId);
try (Git git = this.getGitInstance(repoDir)) {
git.branchCreate()
.setName(branchName)
.setStartPoint(ref)
.call();
return this.infoBranch(workspaceId, branchName);
} catch (GitAPIException e) {
log.error(e, "[newBranch] [error] [workspaceId: {}] [branchName: {}] [ref: {}]", workspaceId, branchName, ref);
throw new ASWException(RCode.ERROR);
}
}
@Override
public void deleteBranch(String workspaceId, String branchName) {
File repoDir = this.getRepoDirPath(workspaceId);
try (Git git = this.getGitInstance(repoDir)) {
git.branchDelete()
.setBranchNames(branchName)
.call();
} catch (GitAPIException e) {
log.error(e, "[deleteBranch] [error] [workspaceId: {}] [branchName: {}]", workspaceId, branchName);
throw new ASWException(RCode.ERROR);
}
}
}

View File

@@ -3,24 +3,26 @@ package net.geedge.asw.module.app.service.impl;
import cn.dev33.satoken.stp.StpUtil; import cn.dev33.satoken.stp.StpUtil;
import cn.hutool.core.io.FileUtil; import cn.hutool.core.io.FileUtil;
import cn.hutool.log.Log; import cn.hutool.log.Log;
import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page; import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl; import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;
import net.geedge.asw.common.util.ASWException; import net.geedge.asw.common.config.Query;
import net.geedge.asw.common.util.Constants; import net.geedge.asw.common.util.*;
import net.geedge.asw.common.util.RCode;
import net.geedge.asw.common.util.T;
import net.geedge.asw.module.app.dao.PackageDao; import net.geedge.asw.module.app.dao.PackageDao;
import net.geedge.asw.module.app.entity.PackageEntity; import net.geedge.asw.module.app.entity.PackageEntity;
import net.geedge.asw.module.app.service.IPackageService; import net.geedge.asw.module.app.service.IPackageService;
import net.geedge.asw.module.app.util.ApkInfo; import net.geedge.asw.module.app.util.ApkInfo;
import net.geedge.asw.module.app.util.ApkUtil; import net.geedge.asw.module.app.util.ApkUtil;
import net.geedge.asw.module.app.util.PkgConstant; import net.geedge.asw.module.app.util.PkgConstant;
import net.geedge.asw.module.job.entity.JobEntity;
import net.geedge.asw.module.job.service.IJobService;
import net.geedge.asw.module.sys.entity.SysUserEntity; import net.geedge.asw.module.sys.entity.SysUserEntity;
import net.geedge.asw.module.sys.service.ISysUserService; import net.geedge.asw.module.sys.service.ISysUserService;
import net.geedge.asw.module.workbook.service.IWorkbookResourceService; import net.geedge.asw.module.workbook.service.IWorkbookResourceService;
import net.geedge.asw.module.workbook.util.WorkbookConstant; import net.geedge.asw.module.workbook.util.WorkbookConstant;
import org.apache.commons.io.FileUtils; import org.apache.commons.io.FileUtils;
import org.springframework.beans.factory.annotation.Autowired; import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.core.io.Resource; import org.springframework.core.io.Resource;
import org.springframework.stereotype.Service; import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional; import org.springframework.transaction.annotation.Transactional;
@@ -41,6 +43,12 @@ public class PackageServiceImpl extends ServiceImpl<PackageDao, PackageEntity> i
@Autowired @Autowired
private IWorkbookResourceService workbookResourceService; private IWorkbookResourceService workbookResourceService;
@Autowired
private IJobService jobService;
@Value("${asw.resources.path:resources}")
private String resources;
@Override @Override
public PackageEntity queryInfo(String id) { public PackageEntity queryInfo(String id) {
PackageEntity entity = this.getById(id); PackageEntity entity = this.getById(id);
@@ -60,7 +68,7 @@ public class PackageServiceImpl extends ServiceImpl<PackageDao, PackageEntity> i
@Override @Override
public Page queryList(Map<String, Object> params) { public Page queryList(Map<String, Object> params) {
Page page = T.PageUtil.getPage(params); Page page = new Query(PackageEntity.class).getPage(params);
List<PackageEntity> packageList = this.getBaseMapper().queryList(page, params); List<PackageEntity> packageList = this.getBaseMapper().queryList(page, params);
page.setRecords(packageList); page.setRecords(packageList);
return page; return page;
@@ -78,7 +86,7 @@ public class PackageServiceImpl extends ServiceImpl<PackageDao, PackageEntity> i
throw new ASWException(RCode.PACKAGE_FILE_TYPE_ERROR); throw new ASWException(RCode.PACKAGE_FILE_TYPE_ERROR);
} }
String saveFileName = pkgId + "." + suffix; String saveFileName = pkgId + "." + suffix;
File destination = T.FileUtil.file(PkgConstant.APK_FILES_DIR, saveFileName); File destination = FileResourceUtil.createFile(resources, workspaceId, Constants.FileTypeEnum.PACKAGE.getType(), pkgId, saveFileName);
PackageEntity entity = new PackageEntity(); PackageEntity entity = new PackageEntity();
ApkUtil apkUtil = new ApkUtil(); ApkUtil apkUtil = new ApkUtil();
apkUtil.setAaptToolPath(Path.of(T.WebPathUtil.getRootPath(), "lib", "aapt").toString()); apkUtil.setAaptToolPath(Path.of(T.WebPathUtil.getRootPath(), "lib", "aapt").toString());
@@ -90,6 +98,7 @@ public class PackageServiceImpl extends ServiceImpl<PackageDao, PackageEntity> i
if (T.ObjectUtil.isNull(apkInfo)) { if (T.ObjectUtil.isNull(apkInfo)) {
throw new ASWException(RCode.PACKAGE_FILE_TYPE_ERROR); throw new ASWException(RCode.PACKAGE_FILE_TYPE_ERROR);
} }
entity.setName(apkInfo.getLabel());
entity.setVersion(apkInfo.getVersionName()); entity.setVersion(apkInfo.getVersionName());
entity.setIdentifier(apkInfo.getPackageName()); entity.setIdentifier(apkInfo.getPackageName());
} else { } else {
@@ -97,7 +106,8 @@ public class PackageServiceImpl extends ServiceImpl<PackageDao, PackageEntity> i
if (T.ObjectUtil.isNull(apkInfo)) { if (T.ObjectUtil.isNull(apkInfo)) {
throw new ASWException(RCode.PACKAGE_FILE_TYPE_ERROR); throw new ASWException(RCode.PACKAGE_FILE_TYPE_ERROR);
} }
entity.setVersion(apkInfo.getSdkVersion()); entity.setName(apkInfo.getLabel());
entity.setVersion(apkInfo.getVersionName());
entity.setIdentifier(apkInfo.getPackageName()); entity.setIdentifier(apkInfo.getPackageName());
} }
} catch (Exception e) { } catch (Exception e) {
@@ -106,7 +116,6 @@ public class PackageServiceImpl extends ServiceImpl<PackageDao, PackageEntity> i
throw new ASWException(RCode.PACKAGE_FILE_TYPE_ERROR); throw new ASWException(RCode.PACKAGE_FILE_TYPE_ERROR);
} }
entity.setId(pkgId); entity.setId(pkgId);
entity.setName(fileResource.getFilename());
entity.setDescription(T.StrUtil.emptyToDefault(description, "")); entity.setDescription(T.StrUtil.emptyToDefault(description, ""));
entity.setPlatform(PkgConstant.Platform.ANDROID.getValue()); entity.setPlatform(PkgConstant.Platform.ANDROID.getValue());
entity.setWorkspaceId(workspaceId); entity.setWorkspaceId(workspaceId);
@@ -123,6 +132,12 @@ public class PackageServiceImpl extends ServiceImpl<PackageDao, PackageEntity> i
@Override @Override
@Transactional(rollbackFor = Exception.class) @Transactional(rollbackFor = Exception.class)
public void removePackage(List<String> ids) { public void removePackage(List<String> ids) {
List<JobEntity> jobList = jobService.list(new LambdaQueryWrapper<JobEntity>().in(JobEntity::getPackageId, ids));
if (T.CollUtil.isNotEmpty(jobList)) {
throw new ASWException(RCode.PACKAGE_CANNOT_DELETE);
}
for (String id : ids) { for (String id : ids) {
PackageEntity entity = this.getById(id); PackageEntity entity = this.getById(id);
// remove file // remove file
@@ -136,4 +151,18 @@ public class PackageServiceImpl extends ServiceImpl<PackageDao, PackageEntity> i
workbookResourceService.removeResource(ids, WorkbookConstant.ResourceType.PACKAGE.getValue()); workbookResourceService.removeResource(ids, WorkbookConstant.ResourceType.PACKAGE.getValue());
} }
@Override
public PackageEntity updatePackage(String workspaceId, String packageId, String name, String description) {
PackageEntity entity = this.getById(packageId);
if (T.StrUtil.isNotEmpty(name)){
entity.setName(name);
}
if (T.StrUtil.isNotEmpty(description)){
entity.setDescription(description);
}
entity.setUpdateTimestamp(System.currentTimeMillis());
entity.setUpdateUserId(StpUtil.getLoginIdAsString());
this.updateById(entity);
return entity;
}
} }

View File

@@ -1,40 +1,31 @@
package net.geedge.asw.module.app.service.impl; package net.geedge.asw.module.app.service.impl;
import cn.dev33.satoken.stp.StpUtil;
import cn.hutool.core.lang.Validator; import cn.hutool.core.lang.Validator;
import cn.hutool.json.JSONArray; import cn.hutool.json.JSONArray;
import cn.hutool.json.JSONObject; import cn.hutool.json.JSONObject;
import cn.hutool.log.Log; import cn.hutool.log.Log;
import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
import net.geedge.asw.common.util.T; import net.geedge.asw.common.util.T;
import net.geedge.asw.module.app.entity.ApplicationEntity; import net.geedge.asw.module.app.entity.ApplicationEntity;
import net.geedge.asw.module.app.entity.ApplicationSignatureEntity;
import net.geedge.asw.module.app.service.IApplicationService;
import net.geedge.asw.module.app.service.IApplicationSignatureService;
import net.geedge.asw.module.app.service.ITSGApplicationService; import net.geedge.asw.module.app.service.ITSGApplicationService;
import net.geedge.asw.module.attribute.entity.AttributeEntity; import net.geedge.asw.module.attribute.entity.AttributeEntity;
import net.geedge.asw.module.attribute.service.IAttributeService; import net.geedge.asw.module.attribute.service.IAttributeService;
import org.springframework.beans.factory.annotation.Autowired; import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service; import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
import java.util.*; import java.util.Collection;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors; import java.util.stream.Collectors;
@Service @Service("tsg2402ApplicationService")
public class TSGApplicationServiceImpl implements ITSGApplicationService { public class TSG2402ApplicationServiceImpl implements ITSGApplicationService {
private static final Log log = Log.get(); private static final Log log = Log.get();
@Autowired @Autowired
private IAttributeService attributeService; private IAttributeService attributeService;
@Autowired
private IApplicationService applicationService;
@Autowired
private IApplicationSignatureService applicationSignatureService;
@Override @Override
public Map<Object, Object> aswToTsg2402(List<ApplicationEntity> appList) { public Map<Object, Object> aswToTsg2402(List<ApplicationEntity> appList) {
@@ -63,12 +54,11 @@ public class TSGApplicationServiceImpl implements ITSGApplicationService {
Map properties = (Map) app.getProperties(); Map properties = (Map) app.getProperties();
Map<Object, Object> app_properties = T.MapUtil.builder() Map<Object, Object> app_properties = T.MapUtil.builder()
.put("parent_app_id", 0) .put("parent_app_id", 0)
.put("parent_app_name", "null") .put("parent_app_name", T.MapUtil.getStr(properties, "parentApp", ""))
.put("category", T.MapUtil.getStr(properties, "category", "")) .put("category", T.MapUtil.getStr(properties, "category", ""))
.put("subcategory", T.MapUtil.getStr(properties, "subcategory", "")) .put("subcategory", T.MapUtil.getStr(properties, "subcategory", ""))
.put("content", T.MapUtil.getStr(properties, "content", "")) .put("content", T.MapUtil.getStr(properties, "content", ""))
.put("risk", T.MapUtil.getStr(properties, "risk", "1")) .put("risk", T.MapUtil.getStr(properties, "risk", "1"))
.put("characteristics", T.MapUtil.getStr(properties, "characteristics", ""))
.put("deny_action", T.MapUtil.builder() .put("deny_action", T.MapUtil.builder()
.put("method", "drop") .put("method", "drop")
.put("after_n_packets", 0) .put("after_n_packets", 0)
@@ -85,8 +75,7 @@ public class TSGApplicationServiceImpl implements ITSGApplicationService {
application.put("app_properties", app_properties); application.put("app_properties", app_properties);
// app_surrogates // app_surrogates
ApplicationSignatureEntity signature = applicationSignatureService.queryLastVersionSignatureByAppId(app.getId()); JSONObject jsonObject = (JSONObject) app.getSignature();
JSONObject jsonObject = T.JSONUtil.parseObj(signature.getContent());
JSONArray surrogates = jsonObject.getJSONArray("surrogates"); JSONArray surrogates = jsonObject.getJSONArray("surrogates");
if (!surrogates.isEmpty()) { if (!surrogates.isEmpty()) {
List<Map> app_surrogates = T.ListUtil.list(true); List<Map> app_surrogates = T.ListUtil.list(true);
@@ -124,8 +113,7 @@ public class TSGApplicationServiceImpl implements ITSGApplicationService {
int sig_object_id = 10, signature_id = 0; int sig_object_id = 10, signature_id = 0;
for (ApplicationEntity app : appList) { for (ApplicationEntity app : appList) {
ApplicationSignatureEntity signature = applicationSignatureService.queryLastVersionSignatureByAppId(app.getId()); JSONObject jsonObject = (JSONObject) app.getSignature();
JSONObject jsonObject = T.JSONUtil.parseObj(signature.getContent());
JSONArray surrogates = jsonObject.getJSONArray("surrogates"); JSONArray surrogates = jsonObject.getJSONArray("surrogates");
List<Object> signaturesForApp = surrogates.stream() List<Object> signaturesForApp = surrogates.stream()
.map(obj -> ((JSONObject) obj).getJSONArray("signatures")) .map(obj -> ((JSONObject) obj).getJSONArray("signatures"))
@@ -146,27 +134,33 @@ public class TSGApplicationServiceImpl implements ITSGApplicationService {
for (Object condition : conditions) { for (Object condition : conditions) {
JSONObject conditionJSONObj = (JSONObject) condition; JSONObject conditionJSONObj = (JSONObject) condition;
String attributeType = T.MapUtil.getStr(conditionJSONObj, "attributeType");
String attributeName = T.MapUtil.getStr(conditionJSONObj, "attributeName"); String attributeName = T.MapUtil.getStr(conditionJSONObj, "attributeName");
AttributeEntity attributeEntity = attributeService.queryAttribute(attributeType, attributeName); AttributeEntity attributeEntity = attributeService.queryAttribute(attributeName);
if (null == attributeEntity || T.StrUtil.isEmpty(attributeEntity.getObjectType())) continue; if (null == attributeEntity || T.StrUtil.isEmpty(attributeEntity.getObjectType())) continue;
Map<Object, Object> or_condition_obj = T.MapUtil.builder() Map<Object, Object> or_condition_obj = T.MapUtil.builder()
.put("lua_profile_id", 0) .put("lua_profile_id", 0)
.put("attribute_type", attributeType) .put("attribute_type", attributeEntity.getType())
.put("attribute_name", attributeName) .put("attribute_name", attributeName)
.put("protocol", attributeEntity.getProtocol()) .put("protocol", attributeEntity.getProtocol())
.build(); .build();
List<Integer> source_object_ids = T.ListUtil.list(true); List<Integer> source_object_ids = T.ListUtil.list(true);
// sig_objects // objects
JSONArray items = conditionJSONObj.getJSONArray("items"); JSONArray objects = conditionJSONObj.getJSONArray("objects");
for (Object tempObject : objects) {
JSONObject tempJsonObject = (JSONObject) tempObject;
String conditionType = attributeEntity.getObjectType(); // sig_objects
if ("application".equalsIgnoreCase(conditionType)) { JSONArray items = tempJsonObject.getJSONArray("items");
String objectName = tempJsonObject.getStr("name");
String objectType = tempJsonObject.getStr("type");
String objectDescription = tempJsonObject.getStr("description");
if ("application".equalsIgnoreCase(objectType)) {
continue; continue;
} else if ("boolean".equals(conditionType)) { } else if ("boolean".equals(objectType)) {
items.stream() items.stream()
.map(obj -> (JSONObject) obj) .map(obj -> (JSONObject) obj)
.forEach(item -> { .forEach(item -> {
@@ -177,7 +171,7 @@ public class TSGApplicationServiceImpl implements ITSGApplicationService {
source_object_ids.add(3); source_object_ids.add(3);
} }
}); });
} else if ("ip_protocol".equals(conditionType)) { } else if ("ip_protocol".equals(objectType)) {
items.stream() items.stream()
.map(obj -> (JSONObject) obj) .map(obj -> (JSONObject) obj)
.forEach(item -> { .forEach(item -> {
@@ -191,30 +185,31 @@ public class TSGApplicationServiceImpl implements ITSGApplicationService {
} }
}); });
} else { } else {
String name = T.MapUtil.getStr((JSONObject) items.getFirst(), "item");
Map<Object, Object> sig_object = T.MapUtil.builder() Map<Object, Object> sig_object = T.MapUtil.builder()
.put("id", sig_object_id) .put("id", sig_object_id)
.put("source_id", sig_object_id) .put("source_id", sig_object_id)
.put("name", name) .put("name", objectName)
.put("source_name", name) .put("source_name", objectName)
.put("type", conditionType) .put("type", objectType)
.put("sub_type", attributeEntity.getType()) .put("sub_type", attributeEntity.getType())
.put("member_type", "item") .put("member_type", "item")
.put("uuid", T.IdUtil.fastSimpleUUID()) .put("uuid", T.IdUtil.fastSimpleUUID())
.put("statistics_option", "none") .put("statistics_option", "none")
.put("description", objectDescription)
.build(); .build();
Map<Object, Object> member = this.buildTSG2402SignaturesMember(attributeEntity, items); Map<Object, Object> member = this.buildTSG2402SignaturesMember(objectType.toLowerCase(), items);
sig_object.put("member", member); sig_object.put("member", member);
sig_objects.add(sig_object); sig_objects.add(sig_object);
source_object_ids.add(sig_object_id); source_object_ids.add(sig_object_id);
sig_object_id++; sig_object_id++;
} }
}
or_condition_obj.put("source_object_ids", source_object_ids); or_condition_obj.put("source_object_ids", source_object_ids);
Map<Object, Object> and_condition_item = T.MapUtil.builder() Map<Object, Object> and_condition_item = T.MapUtil.builder()
.put("not_flag", T.MapUtil.getBool(conditionJSONObj, "negate_option", false) ? 1 : 0) .put("not_flag", T.MapUtil.getBool(conditionJSONObj, "negateOption", false) ? 1 : 0)
.put("or_conditions", T.ListUtil.of(or_condition_obj)) .put("or_conditions", T.ListUtil.of(or_condition_obj))
.build(); .build();
and_conditions.add(and_condition_item); and_conditions.add(and_condition_item);
@@ -303,12 +298,11 @@ public class TSGApplicationServiceImpl implements ITSGApplicationService {
return m; return m;
} }
private Map<Object, Object> buildTSG2402SignaturesMember(AttributeEntity attributeEntity, JSONArray itemArr) { private Map<Object, Object> buildTSG2402SignaturesMember(String objectType, JSONArray itemArr) {
List<Object> list = T.ListUtil.list(true); List<Object> list = T.ListUtil.list(true);
itemArr.stream() itemArr.stream()
.map(obj -> (JSONObject) obj) .map(obj -> (JSONObject) obj)
.forEach(item -> { .forEach(item -> {
String objectType = attributeEntity.getObjectType().toLowerCase();
switch (objectType) { switch (objectType) {
case "keywords": case "keywords":
case "http_signature": { case "http_signature": {
@@ -484,7 +478,6 @@ public class TSGApplicationServiceImpl implements ITSGApplicationService {
// ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ // ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
@Override @Override
@Transactional(rollbackFor = Exception.class)
public List<ApplicationEntity> tsg2402ToAsw(String workspaceId, List<JSONObject> dataList) { public List<ApplicationEntity> tsg2402ToAsw(String workspaceId, List<JSONObject> dataList) {
List<ApplicationEntity> records = T.ListUtil.list(true); List<ApplicationEntity> records = T.ListUtil.list(true);
for (JSONObject tsgAppSourceData : dataList) { for (JSONObject tsgAppSourceData : dataList) {
@@ -498,45 +491,40 @@ public class TSGApplicationServiceImpl implements ITSGApplicationService {
.forEach(application -> { .forEach(application -> {
// application // application
String app_name = application.getStr("app_name"); String app_name = application.getStr("app_name");
String app_longname = application.getStr("app_longname");
String description = application.getStr("description"); String description = application.getStr("description");
JSONObject appProperties = application.getJSONObject("app_properties"); JSONObject appProperties = application.getJSONObject("app_properties");
String category = T.MapUtil.getStr(appProperties, "category", ""); String category = T.MapUtil.getStr(appProperties, "category", "");
String subcategory = T.MapUtil.getStr(appProperties, "subcategory", ""); String subcategory = T.MapUtil.getStr(appProperties, "subcategory", "");
String content = T.MapUtil.getStr(appProperties, "content", ""); String content = T.MapUtil.getStr(appProperties, "content", "");
String parent_app_name = T.MapUtil.getStr(appProperties, "parent_app_name", "");
int risk = T.MapUtil.getInt(appProperties, "risk", 1); int risk = T.MapUtil.getInt(appProperties, "risk", 1);
String characteristics = T.MapUtil.getStr(appProperties, "characteristics", "");
Map<Object, Object> properties = T.MapUtil.builder() Map<Object, Object> properties = T.MapUtil.builder()
.put("category", category) .put("category", category)
.put("subcategory", subcategory) .put("subcategory", subcategory)
.put("content", content) .put("content", content)
.put("parentApp", parent_app_name)
.put("risk", risk) .put("risk", risk)
.put("characteristics", characteristics)
.build(); .build();
// save or update application // meta.json
ApplicationEntity entity = new ApplicationEntity(); ApplicationEntity entity = new ApplicationEntity();
entity.setName(app_name); entity.setName(app_name);
entity.setLongName(app_longname);
entity.setDescription(description); entity.setDescription(description);
entity.setProperties(properties); entity.setProperties(properties);
entity.setPackageName("{}"); // default value
entity.setWorkspaceId(workspaceId); entity.setId(T.StrUtil.uuid());
entity.setCreateTimestamp(System.currentTimeMillis()); entity.setDeveloper("");
entity.setUpdateTimestamp(System.currentTimeMillis()); entity.setWebsite("");
entity.setCreateUserId(StpUtil.getLoginIdAsString()); entity.setPackageName(
entity.setUpdateUserId(StpUtil.getLoginIdAsString()); T.MapUtil.builder()
.put("android", "")
ApplicationEntity one = applicationService.getOne(new LambdaQueryWrapper<ApplicationEntity>() .put("ios", "")
.eq(ApplicationEntity::getWorkspaceId, workspaceId) .build()
.eq(ApplicationEntity::getName, app_name)); );
if (null != one) {
entity.setId(one.getId());
}
applicationService.saveOrUpdate(entity);
records.add(entity);
String applicationId = entity.getId();
// surrogate - signature // surrogate - signature
Map<String, List<String>> surrAndSignListMap = T.MapUtil.newHashMap(); Map<String, List<String>> surrAndSignListMap = T.MapUtil.newHashMap();
@@ -549,7 +537,6 @@ public class TSGApplicationServiceImpl implements ITSGApplicationService {
} }
} }
List<Object> insertSurrogateList = T.ListUtil.list(true); List<Object> insertSurrogateList = T.ListUtil.list(true);
for (Map.Entry<String, List<String>> entry : surrAndSignListMap.entrySet()) { for (Map.Entry<String, List<String>> entry : surrAndSignListMap.entrySet()) {
String surrogateName = entry.getKey(); String surrogateName = entry.getKey();
@@ -575,19 +562,11 @@ public class TSGApplicationServiceImpl implements ITSGApplicationService {
.put("surrogates", insertSurrogateList) .put("surrogates", insertSurrogateList)
.build(); .build();
// save application signatrue // signature.json
ApplicationSignatureEntity signatureEntity = new ApplicationSignatureEntity(); entity.setSignature(T.JSONUtil.parseObj(sm));
signatureEntity.setApplicationId(applicationId);
signatureEntity.setContent(T.JSONUtil.toJsonStr(sm));
signatureEntity.setCreateTimestamp(System.currentTimeMillis());
signatureEntity.setCreateUserId(StpUtil.getLoginIdAsString());
signatureEntity.setOpVersion(0L);
ApplicationSignatureEntity signatureLast = applicationSignatureService.queryLastVersionSignatureByAppId(applicationId); // add records
if (T.ObjectUtil.isNotEmpty(signatureLast)) { records.add(entity);
signatureEntity.setOpVersion(signatureLast.getOpVersion() + 1);
}
applicationSignatureService.save(signatureEntity);
}); });
} }
return records; return records;
@@ -622,12 +601,10 @@ public class TSGApplicationServiceImpl implements ITSGApplicationService {
JSONObject or_condition = (JSONObject) T.JSONUtil.getByPath(conditions, "or_conditions[0]"); JSONObject or_condition = (JSONObject) T.JSONUtil.getByPath(conditions, "or_conditions[0]");
String attribute_name = or_condition.getStr("attribute_name", ""); String attribute_name = or_condition.getStr("attribute_name", "");
String attribute_type = or_condition.getStr("attribute_type", "");
Map<Object, Object> m = T.MapUtil.builder() Map<Object, Object> m = T.MapUtil.builder()
.put("attributeName", attribute_name) .put("attributeName", attribute_name)
.put("attributeType", attribute_type) .put("negateOption", not_flag == 1 ? true : false)
.put("negate_option", not_flag == 1 ? true : false)
.put("description", "") .put("description", "")
.build(); .build();
@@ -642,20 +619,10 @@ public class TSGApplicationServiceImpl implements ITSGApplicationService {
}) })
.collect(Collectors.toList()); .collect(Collectors.toList());
List<Map<Object, Object>> itemList = this.buildAswConditionItemsFromTSG2402(sourceObjectList); List<Map<Object, Object>> objects = this.buildAswConditionObjectsFromTSG2402(sourceObjectList);
if (T.CollUtil.isEmpty(itemList)) continue; if (T.CollUtil.isEmpty(objects)) continue;
// item value 去重 m.put("objects", objects);
List<Map<String, String>> distinctItemList = itemList.stream()
.collect(Collectors.collectingAndThen(
Collectors.toMap(
map -> map.get("item"),
map -> map,
(existing, replacement) -> existing // 保留第一个出现的元素
),
map -> new ArrayList(map.values())
));
m.put("items", distinctItemList);
conditionMapList.add(m); conditionMapList.add(m);
} }
@@ -667,14 +634,17 @@ public class TSGApplicationServiceImpl implements ITSGApplicationService {
return surrogate; return surrogate;
} }
private List<Map<Object, Object>> buildAswConditionItemsFromTSG2402(List<JSONObject> sourceObjectList) { private List<Map<Object, Object>> buildAswConditionObjectsFromTSG2402(List<JSONObject> sourceObjectList) {
List<Map<Object, Object>> iiemList = T.ListUtil.list(true); List<Map<Object, Object>> objectList = T.ListUtil.list(true);
for (JSONObject jsonObject : sourceObjectList) { for (JSONObject jsonObject : sourceObjectList) {
String name = jsonObject.getStr("name");
String type = jsonObject.getStr("type"); String type = jsonObject.getStr("type");
String description = jsonObject.getStr("description");
JSONArray itemArr = (JSONArray) T.JSONUtil.getByPath(jsonObject, "member.items"); JSONArray itemArr = (JSONArray) T.JSONUtil.getByPath(jsonObject, "member.items");
itemArr = T.CollUtil.defaultIfEmpty(itemArr, new JSONArray()); itemArr = T.CollUtil.defaultIfEmpty(itemArr, new JSONArray());
List<Map<Object, Object>> items = T.ListUtil.list(true);
switch (type) { switch (type) {
case "http_signature": case "http_signature":
case "keywords": { case "keywords": {
@@ -727,7 +697,7 @@ public class TSGApplicationServiceImpl implements ITSGApplicationService {
if (T.StrUtil.isNotEmpty(context_name)) { if (T.StrUtil.isNotEmpty(context_name)) {
m.put("district", context_name); m.put("district", context_name);
} }
iiemList.add(m); items.add(m);
}); });
break; break;
} }
@@ -737,7 +707,7 @@ public class TSGApplicationServiceImpl implements ITSGApplicationService {
.map(obj -> (JSONObject) obj) .map(obj -> (JSONObject) obj)
.forEach(item -> { .forEach(item -> {
String str = (String) T.JSONUtil.getByPath(item, "string.patterns[0].keywords"); String str = (String) T.JSONUtil.getByPath(item, "string.patterns[0].keywords");
iiemList.add( items.add(
T.MapUtil.builder() T.MapUtil.builder()
.put("item", str) .put("item", str)
.put("description", "") .put("description", "")
@@ -761,7 +731,7 @@ public class TSGApplicationServiceImpl implements ITSGApplicationService {
if (!"0-65535" .equalsIgnoreCase(port)) { if (!"0-65535" .equalsIgnoreCase(port)) {
ipAddress = T.StrUtil.concat(true, ipAddress, "#", port); ipAddress = T.StrUtil.concat(true, ipAddress, "#", port);
} }
iiemList.add( items.add(
T.MapUtil.builder() T.MapUtil.builder()
.put("item", ipAddress) .put("item", ipAddress)
.put("description", "") .put("description", "")
@@ -778,7 +748,7 @@ public class TSGApplicationServiceImpl implements ITSGApplicationService {
if (T.StrUtil.isEmpty(port)) { if (T.StrUtil.isEmpty(port)) {
port = (String) T.JSONUtil.getByPath(item, "port.port_range"); port = (String) T.JSONUtil.getByPath(item, "port.port_range");
} }
iiemList.add( items.add(
T.MapUtil.builder() T.MapUtil.builder()
.put("item", port) .put("item", port)
.put("description", "") .put("description", "")
@@ -797,7 +767,7 @@ public class TSGApplicationServiceImpl implements ITSGApplicationService {
.put("item", low_boundary + "-" + up_boundary) .put("item", low_boundary + "-" + up_boundary)
.put("description", "") .put("description", "")
.build(); .build();
iiemList.add(m); items.add(m);
}); });
break; break;
} }
@@ -807,7 +777,7 @@ public class TSGApplicationServiceImpl implements ITSGApplicationService {
.put("item", jsonObject.getStr("name")) .put("item", jsonObject.getStr("name"))
.put("description", "") .put("description", "")
.build(); .build();
iiemList.add(m); items.add(m);
break; break;
} }
case "application": { case "application": {
@@ -816,8 +786,28 @@ public class TSGApplicationServiceImpl implements ITSGApplicationService {
default: default:
break; break;
} }
Map<Object, Object> m = T.MapUtil.builder()
.put("name", name)
.put("type", type)
.put("description", description)
.put("items", items)
.build();
objectList.add(m);
} }
return iiemList; return objectList;
}
@Override
public Map<Object, Object> aswToTsg2410(List<ApplicationEntity> appList) {
// 不在这个类处理 TSG2410ApplicationServiceImpl 实现
return null;
}
@Override
public List<ApplicationEntity> tsg2410ToAsw(String workspaceId, List<JSONObject> dataList) {
// 不在这个类处理 TSG2410ApplicationServiceImpl 实现
return null;
} }
} }

View File

@@ -0,0 +1,194 @@
package net.geedge.asw.module.app.service.impl;
import cn.dev33.satoken.stp.StpUtil;
import cn.hutool.log.Log;
import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
import net.geedge.asw.common.util.ASWException;
import net.geedge.asw.common.util.RCode;
import net.geedge.asw.common.util.T;
import net.geedge.asw.module.app.entity.ApplicationReleaseEntity;
import net.geedge.asw.module.app.service.IApplicationReleaseService;
import net.geedge.asw.module.app.service.ITagService;
import net.geedge.asw.module.app.util.JGitUtils;
import net.geedge.asw.module.sys.entity.SysUserEntity;
import net.geedge.asw.module.sys.service.ISysUserService;
import net.geedge.asw.module.workspace.service.IWorkspaceService;
import org.eclipse.jgit.api.Git;
import org.eclipse.jgit.api.errors.GitAPIException;
import org.eclipse.jgit.lib.PersonIdent;
import org.eclipse.jgit.lib.Ref;
import org.eclipse.jgit.lib.Repository;
import org.eclipse.jgit.revwalk.RevCommit;
import org.eclipse.jgit.revwalk.RevTag;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import java.io.File;
import java.io.IOException;
import java.util.Comparator;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
@Service
public class TagServiceImpl implements ITagService {
private final static Log log = Log.get();
@Autowired
private ISysUserService userService;
@Autowired
private IWorkspaceService workspaceService;
@Autowired
private IApplicationReleaseService releaseService;
private String getShortTagName(String name) {
return name.startsWith(JGitUtils.R_TAGS) ? name.replaceAll(JGitUtils.R_TAGS, "") : name;
}
@Override
public Map<Object, Object> infoTag(String workspaceId, String name) {
File gitDir = workspaceService.getGitDir(workspaceId);
try (Repository repository = JGitUtils.openRepository(gitDir);) {
Map<Object, Object> infoTag = this.infoTag(repository, name);
return infoTag;
} catch (IOException e) {
log.error(e, "[infoTag] [error] [workspaceId: {}] [name: {}]", workspaceId, name);
throw new RuntimeException(e);
}
}
/**
* tag 详情
*
* @param repository
* @param name
* @return
* @throws IOException
*/
private Map<Object, Object> infoTag(Repository repository, String name) throws IOException {
Ref ref = repository.findRef(JGitUtils.getFullTagName(name));
T.VerifyUtil.is(ref).notNull(RCode.GIT_TAG_NOT_FOUND.setParam(name));
// tag
RevTag revTag = JGitUtils.infoTag(repository, ref.getObjectId());
String message = revTag.getFullMessage();
// commit
RevCommit revCommit = JGitUtils.infoCommit(repository, ref.getObjectId().getName());
Map<Object, Object> aswCommitInfo = JGitUtils.buildAswCommitInfo(revCommit);
// release
ApplicationReleaseEntity releaseEntity = releaseService.getOne(new LambdaQueryWrapper<ApplicationReleaseEntity>()
.eq(ApplicationReleaseEntity::getWorkspaceId, repository.getDirectory().getName())
.eq(ApplicationReleaseEntity::getTagName, this.getShortTagName(name))
);
Map<Object, Object> m = T.MapUtil.builder()
.put("name", this.getShortTagName(name))
.put("message", message)
.put("createdAt", revTag.getTaggerIdent().getWhen().getTime())
.put("commit", aswCommitInfo)
.put("release", releaseEntity)
.build();
return m;
}
@Override
public List<Map<Object, Object>> listTag(String workspaceId, String search) {
File gitDir = workspaceService.getGitDir(workspaceId);
try (Repository repository = JGitUtils.openRepository(gitDir);
Git git = Git.open(repository.getDirectory())) {
List<Ref> refList = git.tagList().call();
if (T.StrUtil.isNotEmpty(search)) {
refList = refList.stream()
.filter(ref -> {
String name = ref.getName();
return T.StrUtil.containsIgnoreCase(this.getShortTagName(name), search);
})
.collect(Collectors.toList());
}
List<Map<Object, Object>> list = T.ListUtil.list(true);
for (Ref ref : refList) {
list.add(this.infoTag(repository, ref.getName()));
}
// 默认根据 createdAt 字段排序
list = list.stream()
.sorted(Comparator.comparing(map -> T.MapUtil.getLong((Map) map, "createdAt")).reversed())
.collect(Collectors.toList());
return list;
} catch (IOException | GitAPIException e) {
log.error(e, "[listTag] [error] [workspaceId: {}] [search: {}]", workspaceId, search);
throw new RuntimeException(e);
}
}
@Override
public Map<Object, Object> newTag(String workspaceId, String name, String branch, String message) {
File gitDir = workspaceService.getGitDir(workspaceId);
try (Repository repository = JGitUtils.openRepository(gitDir);
Git git = Git.open(repository.getDirectory())) {
// check tag exists
List<Ref> tagListInDb = git.tagList().call();
Ref checkRefExists = tagListInDb.parallelStream()
.filter(ref -> T.StrUtil.equals(name, this.getShortTagName(ref.getName())))
.findFirst()
.orElse(null);
if (null != checkRefExists) {
throw new ASWException(RCode.GIT_TAG_ALREADY_EXISTS.setParam(name));
}
// check branch exists
if (T.BooleanUtil.negate(
JGitUtils.isBranchExists(repository, branch)
)) {
throw new ASWException(RCode.GIT_MERGE_TARGET_BRANCH_NOT_EXIST.setParam(branch));
}
SysUserEntity loginUser = userService.getById(StpUtil.getLoginIdAsString());
PersonIdent personIdent = JGitUtils.buildPersonIdent(loginUser.getName());
RevCommit branchLatestCommit = JGitUtils.getBranchLatestCommit(repository, branch);
// add tag
git.tag()
.setName(name)
.setMessage(message)
.setTagger(personIdent)
.setObjectId(branchLatestCommit)
.setAnnotated(true)
.setForceUpdate(false)
.call();
// return info
return this.infoTag(repository, name);
} catch (IOException | GitAPIException e) {
log.error(e, "[newTag] [error] [workspaceId: {}] [name: {}] [branch: {}]", workspaceId, name, branch);
throw new RuntimeException(e);
}
}
@Override
public void deleteTag(String workspaceId, String name) {
File gitDir = workspaceService.getGitDir(workspaceId);
try (Repository repository = JGitUtils.openRepository(gitDir);
Git git = Git.open(repository.getDirectory())) {
// del tag
git.tagDelete()
.setTags(name)
.call();
// del release
releaseService.removeRelease(workspaceId, name);
} catch (IOException | GitAPIException e) {
log.error(e, "[deleteTag] [error] [workspaceId: {}] [name: {}]", workspaceId, name);
throw new RuntimeException(e);
}
}
}

View File

@@ -38,11 +38,14 @@ public class ApkInfo {
private String versionCode; private String versionCode;
// 外部版本号 // 外部版本号
private String versionName; private String versionName;
// xapk 包含的 apk
private List<String> splitApks;
public ApkInfo() { public ApkInfo() {
this.features = new ArrayList<>(); this.features = new ArrayList<>();
this.icons = new HashMap<>(); this.icons = new HashMap<>();
this.usesPermissions = new ArrayList<>(); this.usesPermissions = new ArrayList<>();
this.splitApks = new ArrayList<>();
} }
public List<String> getFeatures() { public List<String> getFeatures() {
@@ -161,6 +164,14 @@ public class ApkInfo {
this.versionName = versionName; this.versionName = versionName;
} }
public List<String> getSplitApks() {
return splitApks;
}
public void setSplitApks(List<String> splitApks) {
this.splitApks = splitApks;
}
@Override @Override
public String toString() { public String toString() {
return "ApkInfo [features=" + features + ", icon=" + icon + ", icons=" + icons + ", label=" + label + ", launchableActivity=" + launchableActivity + ", minSdkVersion=" + minSdkVersion + ", packageName=" + packageName + ", sdkVersion=" + sdkVersion + ", size=" + size + ", targetSdkVersion=" + targetSdkVersion + ", usesPermissions=" + usesPermissions + ", versionCode=" + versionCode + ", versionName=" + versionName + "]"; return "ApkInfo [features=" + features + ", icon=" + icon + ", icons=" + icons + ", label=" + label + ", launchableActivity=" + launchableActivity + ", minSdkVersion=" + minSdkVersion + ", packageName=" + packageName + ", sdkVersion=" + sdkVersion + ", size=" + size + ", targetSdkVersion=" + targetSdkVersion + ", usesPermissions=" + usesPermissions + ", versionCode=" + versionCode + ", versionName=" + versionName + "]";

View File

@@ -1,11 +1,11 @@
package net.geedge.asw.module.app.util; package net.geedge.asw.module.app.util;
import cn.hutool.core.io.FileUtil;
import cn.hutool.log.Log; import cn.hutool.log.Log;
import net.geedge.asw.common.util.Constants;
import net.geedge.asw.common.util.T; import net.geedge.asw.common.util.T;
import java.io.*; import java.io.*;
import java.util.Enumeration;
import java.util.List;
import java.util.Map; import java.util.Map;
import java.util.zip.ZipEntry; import java.util.zip.ZipEntry;
import java.util.zip.ZipFile; import java.util.zip.ZipFile;
@@ -84,10 +84,13 @@ public class ApkUtil {
public ApkInfo parseXapk(String xapkPath) { public ApkInfo parseXapk(String xapkPath) {
InputStream inputStream = null; InputStream inputStream = null;
BufferedReader reader = null; BufferedReader reader = null;
File tempFile = null; ZipFile zipFile = null;
try { try {
ZipFile zipFile = new ZipFile(T.FileUtil.file(xapkPath)); zipFile = new ZipFile(T.FileUtil.file(xapkPath));
ZipEntry entry = zipFile.getEntry("manifest.json"); ZipEntry entry = zipFile.getEntry("manifest.json");
if (entry == null){
return null;
}
inputStream = zipFile.getInputStream(entry); inputStream = zipFile.getInputStream(entry);
StringBuilder manifestJson = new StringBuilder(); StringBuilder manifestJson = new StringBuilder();
reader = new BufferedReader(new InputStreamReader(inputStream)); reader = new BufferedReader(new InputStreamReader(inputStream));
@@ -95,29 +98,46 @@ public class ApkUtil {
while ((line = reader.readLine()) != null) { while ((line = reader.readLine()) != null) {
manifestJson.append(line).append("\n"); manifestJson.append(line).append("\n");
} }
Map manifest = T.JSONUtil.toBean(manifestJson.toString(), Map.class); Map manifest = T.JSONUtil.toBean(manifestJson.toString(), Map.class);
ZipEntry packageFile = zipFile.getEntry(T.StrUtil.concat(true, T.MapUtil.getStr(manifest, "package_name"), ".apk")); ApkInfo apkInfo = new ApkInfo();
tempFile = T.FileUtil.file(Constants.TEMP_PATH, packageFile.getName()); this.setXapkInfoProperty(manifest, apkInfo);
FileUtil.writeBytes(zipFile.getInputStream(packageFile).readAllBytes(), tempFile);
ApkInfo apkInfo = this.parseApk(tempFile.getPath()); String packageName = apkInfo.getPackageName();
if (apkInfo == null) { List<String> splitApks = apkInfo.getSplitApks();
if (T.CollUtil.isNotEmpty(splitApks)){
Enumeration<? extends ZipEntry> apkEntries = zipFile.entries();
while (apkEntries.hasMoreElements()) {
ZipEntry apk = apkEntries.nextElement();
if (!apk.getName().endsWith(".apk")){
continue;
}
if (!T.StrUtil.equalsIgnoreCase(packageName + ".apk", apk.getName()) && !splitApks.contains(apk.getName())) {
return null; return null;
} }
if (!T.BooleanUtil.and(apkInfo.getVersionName().equals(T.MapUtil.getStr(manifest, "version_name")), }
apkInfo.getPackageName().equals(T.MapUtil.getStr(manifest, "package_name")))) {
return null;
} }
return apkInfo; return apkInfo;
} catch (Exception e) { } catch (Exception e) {
log.error(e, "[parseXapk] [error] [path: {}]", xapkPath); log.error(e, "[parseXapk] [error] [path: {}]", xapkPath);
return null; return null;
} finally { } finally {
T.FileUtil.del(tempFile);
T.IoUtil.close(inputStream); T.IoUtil.close(inputStream);
T.IoUtil.close(reader); T.IoUtil.close(reader);
T.IoUtil.close(zipFile);
} }
} }
private void setXapkInfoProperty(Map manifest, ApkInfo apkInfo) {
apkInfo.setLabel(T.MapUtil.getStr(manifest, "name"));
apkInfo.setVersionName(T.MapUtil.getStr(manifest, "version_name"));
apkInfo.setPackageName(T.MapUtil.getStr(manifest, "package_name"));
List<Map> splitApkMapList = T.MapUtil.get(manifest, "split_apks", List.class);
List<String> splitApkNames = splitApkMapList.stream().map(x -> T.MapUtil.getStr(x, "file")).toList();
apkInfo.setSplitApks(splitApkNames);
}
private void setApkInfoProperty(ApkInfo apkInfo, String source) { private void setApkInfoProperty(ApkInfo apkInfo, String source) {
if (source.startsWith(APPLICATION)) { if (source.startsWith(APPLICATION)) {
String[] rs = source.split("( icon=')|'"); String[] rs = source.split("( icon=')|'");

View File

@@ -0,0 +1,897 @@
package net.geedge.asw.module.app.util;
import cn.hutool.log.Log;
import net.geedge.asw.common.util.ASWException;
import net.geedge.asw.common.util.RCode;
import net.geedge.asw.common.util.T;
import org.apache.commons.io.FilenameUtils;
import org.eclipse.jgit.api.*;
import org.eclipse.jgit.api.errors.ConcurrentRefUpdateException;
import org.eclipse.jgit.api.errors.GitAPIException;
import org.eclipse.jgit.api.errors.JGitInternalException;
import org.eclipse.jgit.archive.ArchiveFormats;
import org.eclipse.jgit.diff.*;
import org.eclipse.jgit.dircache.DirCache;
import org.eclipse.jgit.dircache.DirCacheBuilder;
import org.eclipse.jgit.dircache.DirCacheEntry;
import org.eclipse.jgit.internal.JGitText;
import org.eclipse.jgit.lib.*;
import org.eclipse.jgit.merge.MergeFormatter;
import org.eclipse.jgit.merge.MergeStrategy;
import org.eclipse.jgit.merge.RecursiveMerger;
import org.eclipse.jgit.merge.ThreeWayMerger;
import org.eclipse.jgit.revwalk.RevCommit;
import org.eclipse.jgit.revwalk.RevTag;
import org.eclipse.jgit.revwalk.RevWalk;
import org.eclipse.jgit.revwalk.filter.RevFilter;
import org.eclipse.jgit.storage.file.FileRepositoryBuilder;
import org.eclipse.jgit.transport.PushResult;
import org.eclipse.jgit.transport.RemoteRefUpdate;
import org.eclipse.jgit.treewalk.CanonicalTreeParser;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.core.env.Environment;
import org.springframework.stereotype.Component;
import java.io.*;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.text.MessageFormat;
import java.util.*;
import java.util.concurrent.TimeUnit;
import java.util.stream.Collectors;
@Component
public class JGitUtils {
private final static Log log = Log.get();
/**
* 本地分支引用前缀
*/
public static final String R_HEADS = "refs/heads/";
public static final String R_TAGS = "refs/tags/";
/**
* 默认分支
*/
public static final String DEFAULT_BRANCH = "main";
private static String textExtensions;
@Autowired
public JGitUtils(Environment environment) {
textExtensions = environment.getProperty("file.extensions.text", "txt,csv,md,html,xml,json,log,bat,py,sh,ini,conf,yaml,yml,properties,toml,java,c,cpp,js,php,ts,go,rb,rtf,tex,rss,xhtml,sql");
}
/**
* 返回 git repository
*
* @param gitDir
* @return
* @throws IOException
*/
public static Repository openRepository(File gitDir) throws IOException {
FileRepositoryBuilder builder = new FileRepositoryBuilder();
Repository repository = builder.setGitDir(gitDir)
.readEnvironment()
.findGitDir()
.build();
return repository;
}
/**
* 初始化仓库
*
* @param repoDir
* @param author
*/
public static void initRepository(File repoDir, String author) {
try (
Git git = Git.init()
.setBare(true)
.setDirectory(repoDir)
.setInitialBranch(DEFAULT_BRANCH)
.call();
Repository repository = git.getRepository();
ObjectInserter inserter = repository.getObjectDatabase().newInserter();
) {
DirCache dirCache = DirCache.newInCore();
DirCacheBuilder builder = dirCache.builder();
ObjectId objectId = JGitUtils.insertBlobFileToDatabase(repository, "".getBytes());
DirCacheEntry entry = JGitUtils.buildDirCacheEntry("README.md", FileMode.REGULAR_FILE, objectId);
builder.add(entry);
builder.finish();
// commit
String message = "Initial commit";
PersonIdent personIdent = new PersonIdent(author, "asw@geedgenetworks.com");
JGitUtils.createCommit(repository, DEFAULT_BRANCH, dirCache.writeTree(inserter), message, personIdent);
} catch (GitAPIException | IOException e) {
log.error(e, "[initRepository] [git init error]");
throw new RuntimeException(e);
}
}
/**
* 返回分支是否存在
*
* @param repository
* @param branch
* @return
* @throws IOException
*/
public static boolean isBranchExists(Repository repository, String branch) throws IOException {
Ref ref = repository.findRef(T.StrUtil.concat(true, R_HEADS, branch));
return null != ref;
}
/**
* 获取提交对象
*
* @param repository
* @param commitId
* @return
* @throws IOException
*/
public static RevCommit infoCommit(Repository repository, String commitId) throws IOException {
try (RevWalk revCommits = new RevWalk(repository);) {
RevCommit commit = revCommits.parseCommit(ObjectId.fromString(commitId));
return commit;
}
}
public static String getFullTagName(String name) {
return name.startsWith(R_TAGS) ? name : T.StrUtil.concat(true, R_TAGS, name);
}
/**
* 获取tag对象
*
* @param repository
* @param objectId
* @return
* @throws IOException
*/
public static RevTag infoTag(Repository repository, ObjectId objectId) throws IOException {
try (RevWalk revWalk = new RevWalk(repository)) {
RevTag revTag = revWalk.parseTag(objectId);
return revTag;
}
}
/**
* 返回分支最新提交
*
* @param repository
* @param branch
* @return
*/
public static RevCommit getBranchLatestCommit(Repository repository, String branch) throws IOException {
RevWalk revWalk = new RevWalk(repository);
RevCommit commit = revWalk.parseCommit(repository.resolve(branch));
return commit;
}
/**
* 获取分支提交记录
*
* @param repository
* @param branch
* @param path
* @return
* @throws GitAPIException
* @throws IOException
*/
public static List<RevCommit> getBranchCommitList(Repository repository, String branch, String path) throws GitAPIException, IOException {
List<RevCommit> resultList = T.ListUtil.list(true);
Git git = Git.open(repository.getDirectory());
ObjectId branchRef = git.getRepository().resolve(branch);
LogCommand command = git.log().add(branchRef);
if (T.StrUtil.isNotEmpty(path)) {
command.addPath(path);
}
Iterable<RevCommit> iterable = command.call();
for (RevCommit commit : iterable) {
resultList.add(commit);
}
return resultList;
}
/**
* 获取父提交
*
* @param repository
* @param branch
* @param commitId
* @return
* @throws IOException
*/
public static RevCommit getParentCommit(Repository repository, String branch, String commitId) throws IOException {
try (RevWalk revWalk = new RevWalk(repository)) {
Ref branchRef = repository.findRef(branch);
revWalk.markStart(revWalk.parseCommit(branchRef.getObjectId()));
RevCommit parentCommit = null;
Iterator<RevCommit> iterator = revWalk.iterator();
while (iterator.hasNext()) {
RevCommit currentCommit = iterator.next();
if (currentCommit.getId().getName().equals(commitId)) {
if (currentCommit.getParentCount() > 0) {
parentCommit = currentCommit.getParent(0);
}
break;
}
}
if (null == parentCommit) {
throw new ASWException(RCode.GIT_PARENT_COMMITID_NOT_FOUND);
}
return parentCommit;
}
}
/**
* Returns the merge base of two commits or null if there is no common ancestry.
* 返回两个提交的合并基础,如果没有共同祖先,则返回 null。
*
* @param repository
* @param commitIdA
* @param commitIdB
* @return the commit id of the merge base or null if there is no common base
*/
public static RevCommit getMergeBaseCommit(Repository repository, ObjectId commitIdA, ObjectId commitIdB) {
try (RevWalk rw = new RevWalk(repository)) {
RevCommit a = rw.lookupCommit(commitIdA);
RevCommit b = rw.lookupCommit(commitIdB);
rw.setRevFilter(RevFilter.MERGE_BASE);
rw.markStart(a);
rw.markStart(b);
RevCommit mergeBase = rw.next();
if (mergeBase == null) {
return null;
}
return mergeBase;
} catch (Exception e) {
log.error(e, "Failed to determine merge base");
throw new RuntimeException(e);
}
}
/**
* commit range
* 沿着 parent(0) 的主路径提交列表,忽略所有分支路径的提交
*
* @param repository
* @param newCommitId
* @param oldCommitId
* @return
* @throws IOException
*/
public static List<RevCommit> listCommitRange(Repository repository, String newCommitId, String oldCommitId) throws IOException {
log.info("[listCommitRange] [begin] [repository: {}] [newCommitId: {}] [oldCommitId: {}]", repository, newCommitId, oldCommitId);
List<RevCommit> commitList = T.ListUtil.list(true);
try (RevWalk revWalk = new RevWalk(repository)) {
RevCommit revCommit = revWalk.parseCommit(repository.resolve(newCommitId));
Set<ObjectId> visitedCommits = new HashSet<>();
while (revCommit != null && !visitedCommits.contains(revCommit.getId())) {
if (oldCommitId != null && revCommit.getId().getName().equals(oldCommitId)) {
break;
}
commitList.add(revCommit);
visitedCommits.add(revCommit.getId());
// 沿着 parent(0) 前进
if (revCommit.getParentCount() > 0) {
revCommit = revWalk.parseCommit(revCommit.getParent(0));
} else {
revCommit = null;
}
}
} finally {
log.info("[listCommitRange] [finshed] [repository: {}] [commits size: {}]", repository, commitList.size());
}
return commitList;
}
/**
* 文件前缀
*
* @param applicationName
* @return
*/
public static String buildFilePrefix(String applicationName) {
return T.StrUtil.concat(true, "applications/", applicationName, "/");
}
/**
* git file save path
*
* @param applicationName
* @param fileName
* @return
*/
public static String buildFilePath(String applicationName, String fileName) {
return T.StrUtil.concat(true, buildFilePrefix(applicationName), fileName);
}
/**
* 构建 DirCacheEntry
*
* @param path
* @param mode
* @param objectId
* @return
*/
public static DirCacheEntry buildDirCacheEntry(String path, FileMode mode, ObjectId objectId) {
DirCacheEntry dirCacheEntry = new DirCacheEntry(path);
dirCacheEntry.setFileMode(mode);
dirCacheEntry.setObjectId(objectId);
return dirCacheEntry;
}
public static PersonIdent buildPersonIdent(String name) {
return buildPersonIdent(name, "asw@geedgenetworks.com");
}
public static PersonIdent buildPersonIdent(String name, String email) {
return new PersonIdent(name, email);
}
/**
* 将 file object 添加到 objectDatabases 并返回 ObjectId
*
* @param repository
* @param data
* @return
* @throws IOException
*/
public static ObjectId insertBlobFileToDatabase(Repository repository, byte[] data) throws IOException {
try (ObjectInserter inserter = repository.getObjectDatabase().newInserter()) {
ObjectId blobId = inserter.insert(Constants.OBJ_BLOB, data);
inserter.flush();
return blobId;
}
}
/**
* commit
*
* @param repository
* @param branch
* @param treeId
* @param message
* @param personIdent
* @throws IOException
* @throws ConcurrentRefUpdateException
*/
public static void createCommit(Repository repository, String branch, ObjectId treeId, String message, PersonIdent personIdent) throws IOException, ConcurrentRefUpdateException {
try (ObjectInserter inserter = repository.getObjectDatabase().newInserter();
RevWalk revWalk = new RevWalk(repository);) {
CommitBuilder builder = new CommitBuilder();
builder.setTreeId(treeId);
builder.setMessage(message);
ObjectId branchRef = repository.resolve(branch);
if (null != branchRef) {
RevCommit parentId = revWalk.parseCommit(branchRef);
builder.setParentId(parentId);
}
builder.setAuthor(personIdent);
builder.setCommitter(personIdent);
// 插入新的提交对象
ObjectId commitId = inserter.insert(builder);
inserter.flush();
// 更新 branch 指向新的提交
RefUpdate ru = null;
RefUpdate.Result rc = null;
if (null != branchRef) {
ru = repository.updateRef(T.StrUtil.concat(true, R_HEADS, branch));
ru.setNewObjectId(commitId);
rc = ru.update();
} else {
ru = repository.updateRef(Constants.HEAD);
ru.setNewObjectId(commitId);
rc = ru.update();
}
switch (rc) {
case NEW:
case FORCED:
case FAST_FORWARD:
break;
case REJECTED:
case LOCK_FAILURE:
throw new ConcurrentRefUpdateException(JGitText.get().couldNotLockHEAD, ru.getRef(), rc);
default:
throw new JGitInternalException(MessageFormat.format(
JGitText.get().updatingRefFailed, branch, commitId.toString(),
rc));
}
}
}
/**
* commit index
*
* @param repository
* @param branch
* @param index
* @param parentId1
* @param parentId2
* @param message
* @param personIdent
* @return
* @throws IOException
* @throws ConcurrentRefUpdateException
*/
public static boolean commitIndex(Repository repository, String branch, DirCache index, ObjectId parentId1, ObjectId parentId2, String message, PersonIdent personIdent) throws IOException, ConcurrentRefUpdateException {
boolean success = false;
try (ObjectInserter odi = repository.newObjectInserter()) {
// new index
ObjectId indexTreeId = index.writeTree(odi);
// PersonIdent
CommitBuilder commit = new CommitBuilder();
commit.setAuthor(personIdent);
commit.setCommitter(personIdent);
if (null != parentId1 && null == parentId2) {
commit.setParentId(parentId1);
}
if (null != parentId1 && null != parentId2) {
commit.setParentIds(parentId1, parentId2);
}
commit.setTreeId(indexTreeId);
commit.setMessage(message);
// insert the commit into the repository
ObjectId commitId = odi.insert(commit);
odi.flush();
RefUpdate ru = repository.updateRef(T.StrUtil.concat(true, R_HEADS, branch));
ru.setNewObjectId(commitId);
RefUpdate.Result rc = ru.update();
switch (rc) {
case NEW:
case FORCED:
case FAST_FORWARD:
success = true;
break;
case REJECTED:
case LOCK_FAILURE:
throw new ConcurrentRefUpdateException(JGitText.get().couldNotLockHEAD, ru.getRef(), rc);
default:
throw new JGitInternalException(MessageFormat.format(
JGitText.get().updatingRefFailed, branch, commitId.toString(),
rc));
}
}
return success;
}
/**
* 获取 commitIdA -> commitIdB 文件差异
*
* @param repository
* @param newCommitId
* @param oldCommitId
* @return
* @throws IOException
*/
public static List<Map<Object, Object>> getDiffFileListInCommits(Repository repository, String newCommitId, String oldCommitId) throws IOException {
log.info("[getDiffFileListInCommits] [begin] [repository: {}] [newCommitId: {}] [oldCommitId: {}]", repository, newCommitId, oldCommitId);
try (RevWalk revWalk = new RevWalk(repository);
DiffFormatter diffFormatter = new DiffFormatter(null);
) {
RevCommit oldCommit = revWalk.parseCommit(repository.resolve(oldCommitId));
RevCommit newCommit = revWalk.parseCommit(repository.resolve(newCommitId));
// oldTree
CanonicalTreeParser oldTree = new CanonicalTreeParser();
oldTree.reset(repository.newObjectReader(), oldCommit.getTree());
// newTree
CanonicalTreeParser newTree = new CanonicalTreeParser();
newTree.reset(repository.newObjectReader(), newCommit.getTree());
// diff
List<Map<Object, Object>> files = T.ListUtil.list(true);
diffFormatter.setRepository(repository);
diffFormatter.setDiffComparator(RawTextComparator.DEFAULT);
diffFormatter.setDetectRenames(true);
List<DiffEntry> diffs = diffFormatter.scan(oldTree, newTree);
for (DiffEntry diff : diffs) {
int addedLines = 0;
int deletedLines = 0;
EditList edits = diffFormatter.toFileHeader(diff).toEditList();
for (Edit edit : edits) {
switch (edit.getType()) {
case INSERT:
addedLines += edit.getLengthB();
break;
case DELETE:
deletedLines += edit.getLengthA();
break;
case REPLACE:
addedLines += edit.getLengthB();
deletedLines += edit.getLengthA();
break;
default:
break;
}
}
String oldPath = diff.getOldPath(), newPath = diff.getNewPath(), encoding = null, oldContent = null, newContent = null;
switch (diff.getChangeType()) {
case COPY:
case ADD: {
Map<Object, Object> fileContent = getFileContent(repository, newPath, diff.getNewId().toObjectId());
encoding = T.MapUtil.getStr(fileContent, "encoding", "");
newContent = T.MapUtil.getStr(fileContent, "content", "");
break;
}
case DELETE: {
Map<Object, Object> fileContent = getFileContent(repository, oldPath, diff.getOldId().toObjectId());
encoding = T.MapUtil.getStr(fileContent, "encoding", "");
oldContent = T.MapUtil.getStr(fileContent, "content", "");
break;
}
case MODIFY: {
Map<Object, Object> fileContent = getFileContent(repository, oldPath, diff.getOldId().toObjectId());
oldContent = T.MapUtil.getStr(fileContent, "content", "");
Map<Object, Object> fileContent1 = getFileContent(repository, newPath, diff.getNewId().toObjectId());
encoding = T.MapUtil.getStr(fileContent1, "encoding", "");
newContent = T.MapUtil.getStr(fileContent1, "content", "");
break;
}
case RENAME: {
if (0 != addedLines | 0 != deletedLines) {
Map<Object, Object> fileContent = getFileContent(repository, oldPath, diff.getOldId().toObjectId());
oldContent = T.MapUtil.getStr(fileContent, "content", "");
Map<Object, Object> fileContent1 = getFileContent(repository, newPath, diff.getNewId().toObjectId());
encoding = T.MapUtil.getStr(fileContent1, "encoding", "");
newContent = T.MapUtil.getStr(fileContent1, "content", "");
}
break;
}
default:
break;
}
files.add(
T.MapUtil.builder()
.put("oldPath", oldPath)
.put("newPath", newPath)
.put("addedLines", addedLines)
.put("removedLines", deletedLines)
.put("encoding", encoding)
.put("oldContent", oldContent)
.put("newContent", newContent)
.put("action", diff.getChangeType().name().toLowerCase())
.build()
);
}
return files;
}
}
/**
* 获取 sourceBranch,targetBranch 冲突文件路径
*
* @param repository
* @param srcBranch
* @param tgtBranch
* @return
* @throws IOException
*/
public static List<String> getConflictFilePathInBranches(Repository repository, String srcBranch, String tgtBranch) throws IOException {
log.info("[getConflictFileListInBranches] [begin] [repository: {}] [srcBranch: {}] [tgtBranch: {}]", repository, srcBranch, tgtBranch);
try (RevWalk revWalk = new RevWalk(repository)) {
RevCommit commitA = revWalk.parseCommit(repository.resolve(srcBranch));
RevCommit commitB = revWalk.parseCommit(repository.resolve(tgtBranch));
// Find the merge base
RevCommit mergeBaseCommit = JGitUtils.getMergeBaseCommit(repository, commitA, commitB);
String mergeBaseId = mergeBaseCommit.getName();
log.info("[getConflictFileListInBranches] [mergeBase: {}]", mergeBaseId);
RevCommit mergeBase = revWalk.parseCommit(repository.resolve(mergeBaseId));
ThreeWayMerger threeWayMerger = (ThreeWayMerger) MergeStrategy.RECURSIVE.newMerger(repository, true);
threeWayMerger.setBase(mergeBase);
boolean isOk = threeWayMerger.merge(commitA, commitB);
log.info("[getConflictFileListInBranches] [merge result] [isOk: {}]", isOk);
if (!isOk) {
List<String> unmergedPaths = ((RecursiveMerger) threeWayMerger).getUnmergedPaths();
return unmergedPaths;
}
return T.ListUtil.empty();
}
}
/**
* 获取 sourceBranch,targetBranch 冲突文件详情
*
* @param repository
* @param srcBranch
* @param tgtBranch
* @return
* @throws IOException
*/
public static List<Map<Object, Object>> getConflictFileInfoInBranches(Repository repository, String srcBranch, String tgtBranch) throws IOException {
log.info("[getConflictFileInfoInBranches] [begin] [repository: {}] [srcBranch: {}] [tgtBranch: {}]", repository, srcBranch, tgtBranch);
try (RevWalk revWalk = new RevWalk(repository);) {
RevCommit commitA = revWalk.parseCommit(repository.resolve(srcBranch));
RevCommit commitB = revWalk.parseCommit(repository.resolve(tgtBranch));
// Find the merge base
String mergeBaseId = getMergeBaseCommit(repository, commitA, commitB).getName();
log.info("[getConflictFileInfoInBranches] [mergeBase: {}]", mergeBaseId);
RevCommit mergeBase = revWalk.parseCommit(repository.resolve(mergeBaseId));
ThreeWayMerger threeWayMerger = (ThreeWayMerger) MergeStrategy.RECURSIVE.newMerger(repository, true);
threeWayMerger.setBase(mergeBase);
boolean isOk = threeWayMerger.merge(commitA, commitB);
log.info("[getConflictFileInfoInBranches] [merge result] [isOk: {}]", isOk);
if (!isOk) {
List<Map<Object, Object>> files = T.ListUtil.list(true);
Map<String, org.eclipse.jgit.merge.MergeResult<? extends Sequence>> mergeResults = ((RecursiveMerger) threeWayMerger).getMergeResults();
for (Map.Entry<String, org.eclipse.jgit.merge.MergeResult<? extends Sequence>> entry : mergeResults.entrySet()) {
String unmergedPath = entry.getKey();
// 暂不支持处理二进制文件冲突
if (isBinary(T.FileNameUtil.getName(unmergedPath))) {
throw new ASWException(RCode.GIT_BINARY_CONFLICT_ERROR);
}
org.eclipse.jgit.merge.MergeResult<? extends Sequence> mergeResult = entry.getValue();
try (ByteArrayOutputStream outputStream = new ByteArrayOutputStream()) {
MergeFormatter formatter = new MergeFormatter();
String oursName = unmergedPath;
String theirsName = unmergedPath;
formatter.formatMerge(outputStream, mergeResult, mergeBaseId, oursName, theirsName, StandardCharsets.UTF_8);
files.add(
T.MapUtil.builder()
.put("path", unmergedPath)
.put("content", outputStream.toString(StandardCharsets.UTF_8.name()))
.put("conflict", true)
.build()
);
}
}
return files;
}
return T.ListUtil.empty();
}
}
/**
* 分支合并
*
* @param centralRepository
* @param srcBranch
* @param tgtBranch
* @param author
* @param message
* @param resolveConflictFileContent
* @throws RuntimeException
* @throws GitAPIException
* @throws IOException
*/
public static void mergeBranch(Repository centralRepository, String srcBranch, String tgtBranch, String author, String message, List<Map<String, String>> resolveConflictFileContent) throws RuntimeException, GitAPIException, IOException {
log.info("[mergeBranch] [begin] [repository: {}] [srcBranch: {}] [tgtBranch: {}]", centralRepository, srcBranch, tgtBranch);
// prepare a new folder for the cloned repository
File localPath = T.FileUtil.file(net.geedge.asw.common.util.Constants.TEMP_PATH, T.StrUtil.uuid());
T.FileUtil.del(localPath);
T.FileUtil.mkdir(localPath);
// bare repository
File repoDir = centralRepository.getDirectory();
// clone
try (Git git = Git.cloneRepository()
.setBare(false)
.setURI(repoDir.getAbsolutePath())
.setDirectory(localPath)
.setCredentialsProvider(null)
.call();
Repository repository = git.getRepository();) {
StoredConfig config = repository.getConfig();
config.setString("user", null, "name", author);
config.setString("user", null, "email", "asw@geedgenetworks.com");
config.save();
// git fetch
git.fetch().call();
// checout
git.checkout()
.setCreateBranch(T.StrUtil.equals(DEFAULT_BRANCH, tgtBranch) ? false : true)
.setName(tgtBranch)
.setUpstreamMode(CreateBranchCommand.SetupUpstreamMode.TRACK)
.setStartPoint("origin/" + tgtBranch)
.call();
// merge
MergeResult mergeResult = git.merge()
.setCommit(true)
.setMessage(message)
.setStrategy(MergeStrategy.RECURSIVE)
.setFastForward(MergeCommand.FastForwardMode.NO_FF)
.include(repository.findRef("origin/" + srcBranch))
.call();
MergeResult.MergeStatus mergeStatus = mergeResult.getMergeStatus();
log.info("[mergeBranch] [merge status: {}]", mergeStatus);
if (!mergeStatus.isSuccessful()) {
// 解决冲突
if (mergeStatus == MergeResult.MergeStatus.CONFLICTING && T.CollUtil.isNotEmpty(resolveConflictFileContent)) {
Map<String, int[][]> conflicts = mergeResult.getConflicts();
for (Map.Entry<String, int[][]> entry : conflicts.entrySet()) {
String conflictFilePath = entry.getKey();
Map<String, String> map = resolveConflictFileContent.stream()
.filter(m -> T.StrUtil.equals(conflictFilePath, T.MapUtil.getStr(m, "path")))
.findFirst()
.orElse(null);
if (null != map) {
Path filePath = Paths.get(localPath.getAbsolutePath(), conflictFilePath);
Files.write(filePath, T.MapUtil.getStr(map, "content").getBytes(StandardCharsets.UTF_8));
git.add().addFilepattern(conflictFilePath).call();
}
}
git.commit().setMessage(message).call();
} else {
// 其他类型的合并错误,抛出异常
String errorMessage = String.format("Merge failed: %s, Conflicts: %s", mergeStatus, mergeResult.getConflicts());
throw new RuntimeException(errorMessage);
}
}
// push
Iterable<PushResult> pushResultIterable = git.push()
.setRemote("origin")
.add(tgtBranch)
.call();
for (PushResult pushResult : pushResultIterable) {
for (RemoteRefUpdate update : pushResult.getRemoteUpdates()) {
if (update.getStatus() != RemoteRefUpdate.Status.OK && update.getStatus() != RemoteRefUpdate.Status.UP_TO_DATE) {
log.error("[mergeBranch] [push error] [remote: {}] [status: {}]", update.getRemoteName(), update.getStatus());
String errorMessage = "Push failed: " + update.getStatus();
throw new RuntimeException(errorMessage);
}
}
}
} finally {
T.FileUtil.del(localPath);
}
}
/**
* 归档
*
* @param repository
* @param tagName
* @param file
* @param prefix
* @param format
* @throws IOException
* @throws GitAPIException
*/
public static void archive(Repository repository, String tagName, File file, String prefix, String format) throws IOException, GitAPIException {
ArchiveFormats.registerAll();
T.FileUtil.mkdir(T.FileUtil.getParent(file, 1));
try (OutputStream out = new FileOutputStream(file)) {
try (Git git = new Git(repository)) {
git.archive()
.setTree(repository.resolve(tagName))
.setFormat(format)
.setPrefix(prefix)
.setOutputStream(out)
.call();
}
} finally {
ArchiveFormats.unregisterAll();
}
}
/**
* build asw commit info
*
* @param commit
* @return
*/
public static Map<Object, Object> buildAswCommitInfo(RevCommit commit) {
if (null == commit) {
return T.MapUtil.newHashMap();
}
Map<Object, Object> m = new LinkedHashMap<>();
m.put("id", commit.getName());
m.put("shortId", T.StrUtil.subPre(commit.getName(), 8));
m.put("createdAt", TimeUnit.SECONDS.toMillis(commit.getCommitTime()));
m.put("title", commit.getShortMessage());
m.put("message", commit.getFullMessage());
List<String> parentIds = Arrays.stream(commit.getParents()).map(RevCommit::getName).collect(Collectors.toList());
m.put("parentIds", parentIds);
PersonIdent authorIdent = commit.getAuthorIdent();
m.put("authorName", authorIdent.getName());
m.put("authorEmail", authorIdent.getEmailAddress());
m.put("authoredDate", authorIdent.getWhen().getTime());
PersonIdent committerIdent = commit.getCommitterIdent();
m.put("committerName", committerIdent.getName());
m.put("committerEmail", committerIdent.getEmailAddress());
m.put("committedDate", committerIdent.getWhen().getTime());
return m;
}
/**
* is binary file
*
* @param filename
* @return
*/
public static boolean isBinary(String filename) {
String extension = FilenameUtils.getExtension(filename);
List<String> split = T.StrUtil.split(textExtensions, ",");
return !split.contains(extension.toLowerCase());
}
/**
* 根据 path,objectId 读取文件内容
* 响应 Map,key=encoding,content
*
* @param repository
* @param path
* @param objectId
* @return
* @throws IOException
*/
public static Map<Object, Object> getFileContent(Repository repository, String path, ObjectId objectId) throws IOException {
String encoding = null, content = null;
ObjectLoader loader = repository.open(objectId);
if (isBinary(T.FileNameUtil.getName(path))) {
encoding = "base64";
content = Base64.getEncoder().encodeToString(loader.getBytes());
} else {
content = T.StrUtil.utf8Str(loader.getBytes());
}
return T.MapUtil.builder()
.put("encoding", encoding)
.put("content", content)
.build();
}
}

View File

@@ -9,5 +9,5 @@ import java.util.Map;
public interface IAttributeService extends IService<AttributeEntity> { public interface IAttributeService extends IService<AttributeEntity> {
Page queryList(Map<String, Object> params); Page queryList(Map<String, Object> params);
AttributeEntity queryAttribute(String type, String name); AttributeEntity queryAttribute(String name);
} }

View File

@@ -3,8 +3,7 @@ package net.geedge.asw.module.attribute.service.impl;
import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper; import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page; import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl; import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;
import net.geedge.asw.common.config.Query;
import net.geedge.asw.common.util.T;
import net.geedge.asw.module.attribute.dao.AttributeDao; import net.geedge.asw.module.attribute.dao.AttributeDao;
import net.geedge.asw.module.attribute.entity.AttributeEntity; import net.geedge.asw.module.attribute.entity.AttributeEntity;
import net.geedge.asw.module.attribute.service.IAttributeService; import net.geedge.asw.module.attribute.service.IAttributeService;
@@ -19,17 +18,15 @@ public class AttributeServiceImpl extends ServiceImpl<AttributeDao, AttributeEnt
@Override @Override
public Page queryList(Map<String, Object> params) { public Page queryList(Map<String, Object> params) {
Page page = new Query(AttributeEntity.class).getPage(params);
Page page = T.PageUtil.getPage(params);
List<AttributeEntity> attributeList = this.getBaseMapper().queryList(params); List<AttributeEntity> attributeList = this.getBaseMapper().queryList(params);
page.setRecords(attributeList); page.setRecords(attributeList);
return page; return page;
} }
@Override @Override
public AttributeEntity queryAttribute(String type, String name) { public AttributeEntity queryAttribute(String name) {
AttributeEntity one = this.getOne(new LambdaQueryWrapper<AttributeEntity>() AttributeEntity one = this.getOne(new LambdaQueryWrapper<AttributeEntity>()
.eq(AttributeEntity::getType, type)
.eq(AttributeEntity::getName, name) .eq(AttributeEntity::getName, name)
.last("limit 1") .last("limit 1")
); );

View File

@@ -18,9 +18,9 @@ import net.geedge.asw.module.environment.entity.EnvironmentSessionEntity;
import net.geedge.asw.module.environment.service.IEnvironmentService; import net.geedge.asw.module.environment.service.IEnvironmentService;
import net.geedge.asw.module.environment.service.IEnvironmentSessionService; import net.geedge.asw.module.environment.service.IEnvironmentSessionService;
import net.geedge.asw.module.environment.util.EnvironmentUtil; import net.geedge.asw.module.environment.util.EnvironmentUtil;
import net.geedge.asw.module.runner.entity.PcapEntity; import net.geedge.asw.module.job.entity.PcapEntity;
import net.geedge.asw.module.runner.service.IPcapService; import net.geedge.asw.module.job.service.IPcapService;
import net.geedge.asw.module.runner.util.RunnerConstant; import net.geedge.asw.module.job.util.JobConstant;
import net.geedge.asw.module.sys.service.ISysUserService; import net.geedge.asw.module.sys.service.ISysUserService;
import net.geedge.asw.module.workspace.entity.WorkspaceEntity; import net.geedge.asw.module.workspace.entity.WorkspaceEntity;
import net.geedge.asw.module.workspace.service.IWorkspaceService; import net.geedge.asw.module.workspace.service.IWorkspaceService;
@@ -28,8 +28,6 @@ import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.*; import org.springframework.http.*;
import org.springframework.transaction.annotation.Transactional; import org.springframework.transaction.annotation.Transactional;
import org.springframework.web.bind.annotation.*; import org.springframework.web.bind.annotation.*;
import org.springframework.web.socket.CloseStatus;
import org.springframework.web.socket.WebSocketSession;
import java.io.File; import java.io.File;
import java.io.FileOutputStream; import java.io.FileOutputStream;
@@ -169,25 +167,7 @@ public class EnvironmentController {
@DeleteMapping("/{envId}/session/{sessionId}") @DeleteMapping("/{envId}/session/{sessionId}")
@Transactional @Transactional
public R removeSession(@PathVariable("envId") String envId, @PathVariable("sessionId") String sessionId, @RequestParam String workspaceId) { public R removeSession(@PathVariable("envId") String envId, @PathVariable("sessionId") String sessionId, @RequestParam String workspaceId) {
EnvironmentSessionEntity session = environmentSessionService.getById(sessionId); environmentService.removeSession(sessionId);
WebSocketSession novncSession = Constants.ENV_NOVNC_WEBSOCKET_SESSION.get(sessionId);
WebSocketSession terminalSession = Constants.ENV_TERMINAL_WEBSOCKET_SESSION.get(sessionId);
// 根据 session 找到 novncSession&terminalSession ,更新状态,设置结束时间
session.setEndTimestamp(System.currentTimeMillis());
session.setStatus(2);
environmentSessionService.updateById(session);
try {
if (T.ObjectUtil.isNotEmpty(novncSession)) {
Constants.ENV_NOVNC_WEBSOCKET_SESSION.remove(sessionId);
novncSession.close(CloseStatus.NORMAL.withReason("Administrator disconnected."));
}
if (T.ObjectUtil.isNotEmpty(terminalSession)) {
Constants.ENV_TERMINAL_WEBSOCKET_SESSION.remove(sessionId);
terminalSession.close(CloseStatus.NORMAL.withReason("Administrator disconnected."));
}
} catch (IOException e) {
log.error(e, "RemoveSession send exit prompt error sessionId: {}", sessionId);
}
return R.ok(); return R.ok();
} }
@@ -229,7 +209,7 @@ public class EnvironmentController {
entity.setId(pcapId); entity.setId(pcapId);
entity.setName(destination.getName()); entity.setName(destination.getName());
entity.setSize(destination.length()); entity.setSize(destination.length());
entity.setStatus(RunnerConstant.PcapStatus.UPLOADED.getValue()); entity.setStatus(JobConstant.PcapStatus.UPLOADED.getValue());
entity.setCreateTimestamp(System.currentTimeMillis()); entity.setCreateTimestamp(System.currentTimeMillis());
entity.setCreateUserId(StpUtil.getLoginIdAsString()); entity.setCreateUserId(StpUtil.getLoginIdAsString());
entity.setWorkspaceId(workspace.getId()); entity.setWorkspaceId(workspace.getId());

View File

@@ -22,4 +22,6 @@ public interface IEnvironmentService extends IService<EnvironmentEntity>{
EnvironmentEntity saveEnv(EnvironmentEntity entity); EnvironmentEntity saveEnv(EnvironmentEntity entity);
EnvironmentEntity updateEnv(EnvironmentEntity entity); EnvironmentEntity updateEnv(EnvironmentEntity entity);
void removeSession(String sessionId);
} }

View File

@@ -7,8 +7,10 @@ import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl; import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;
import net.geedge.asw.common.config.Query; import net.geedge.asw.common.config.Query;
import net.geedge.asw.common.util.ASWException; import net.geedge.asw.common.util.ASWException;
import net.geedge.asw.common.util.Constants;
import net.geedge.asw.common.util.RCode; import net.geedge.asw.common.util.RCode;
import net.geedge.asw.common.util.T; import net.geedge.asw.common.util.T;
import net.geedge.asw.module.attribute.entity.AttributeEntity;
import net.geedge.asw.module.environment.dao.EnvironmentDao; import net.geedge.asw.module.environment.dao.EnvironmentDao;
import net.geedge.asw.module.environment.entity.EnvironmentEntity; import net.geedge.asw.module.environment.entity.EnvironmentEntity;
import net.geedge.asw.module.environment.entity.EnvironmentSessionEntity; import net.geedge.asw.module.environment.entity.EnvironmentSessionEntity;
@@ -23,7 +25,10 @@ import net.geedge.asw.module.workspace.service.IWorkspaceService;
import org.springframework.beans.factory.annotation.Autowired; import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service; import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional; import org.springframework.transaction.annotation.Transactional;
import org.springframework.web.socket.CloseStatus;
import org.springframework.web.socket.WebSocketSession;
import java.io.IOException;
import java.util.List; import java.util.List;
import java.util.Map; import java.util.Map;
import java.util.function.Function; import java.util.function.Function;
@@ -88,7 +93,7 @@ public class EnvironmentServiceImpl extends ServiceImpl<EnvironmentDao, Environm
@Override @Override
public Page queryList(Map<String, Object> params) { public Page queryList(Map<String, Object> params) {
Page page = T.PageUtil.getPage(params); Page page = new Query(EnvironmentEntity.class).getPage(params);
List<EnvironmentEntity> packageList = this.getBaseMapper().queryList(page, params); List<EnvironmentEntity> packageList = this.getBaseMapper().queryList(page, params);
List<EnvironmentSessionEntity> sessionEntityList = environmentSessionService.queryListByUsed(); List<EnvironmentSessionEntity> sessionEntityList = environmentSessionService.queryListByUsed();
@@ -201,4 +206,27 @@ public class EnvironmentServiceImpl extends ServiceImpl<EnvironmentDao, Environm
return entity; return entity;
} }
@Override
public void removeSession(String sessionId) {
EnvironmentSessionEntity session = environmentSessionService.getById(sessionId);
WebSocketSession novncSession = Constants.ENV_NOVNC_WEBSOCKET_SESSION.get(sessionId);
WebSocketSession terminalSession = Constants.ENV_TERMINAL_WEBSOCKET_SESSION.get(sessionId);
// 根据 session 找到 novncSession&terminalSession ,更新状态,设置结束时间
session.setEndTimestamp(System.currentTimeMillis());
session.setStatus(2);
environmentSessionService.updateById(session);
try {
if (T.ObjectUtil.isNotEmpty(novncSession)) {
Constants.ENV_NOVNC_WEBSOCKET_SESSION.remove(sessionId);
novncSession.close(CloseStatus.NORMAL.withReason("Administrator disconnected."));
}
if (T.ObjectUtil.isNotEmpty(terminalSession)) {
Constants.ENV_TERMINAL_WEBSOCKET_SESSION.remove(sessionId);
terminalSession.close(CloseStatus.NORMAL.withReason("Administrator disconnected."));
}
} catch (IOException e) {
log.error(e, "RemoveSession send exit prompt error sessionId: {}", sessionId);
}
}
} }

View File

@@ -0,0 +1,208 @@
package net.geedge.asw.module.job.controller;
import cn.hutool.http.HttpRequest;
import cn.hutool.http.HttpResponse;
import cn.hutool.http.Method;
import cn.hutool.json.JSONObject;
import cn.hutool.log.Log;
import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
import com.baomidou.mybatisplus.core.conditions.update.LambdaUpdateWrapper;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import net.geedge.asw.common.util.*;
import net.geedge.asw.module.environment.entity.EnvironmentEntity;
import net.geedge.asw.module.environment.entity.EnvironmentSessionEntity;
import net.geedge.asw.module.environment.service.IEnvironmentService;
import net.geedge.asw.module.environment.service.IEnvironmentSessionService;
import net.geedge.asw.module.job.entity.JobCfgEntity;
import net.geedge.asw.module.job.entity.JobEntity;
import net.geedge.asw.module.job.service.IJobCfgService;
import net.geedge.asw.module.job.util.JobQueueManager;
import net.geedge.asw.module.job.service.IJobService;
import net.geedge.asw.module.job.util.JobConstant;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.*;
import java.util.Arrays;
import java.util.List;
import java.util.Map;
import java.util.concurrent.TimeUnit;
@RestController
@RequestMapping("/api/v1/workspace")
public class JobController {
private static final Log log = Log.get();
@Autowired
private IJobService jobService;
@Autowired
private IEnvironmentService environmentService;
@Autowired
private IEnvironmentSessionService sessionService;
@Autowired
private JobQueueManager jobQueueManager;
@Autowired
private IJobCfgService jobCfgService;
@GetMapping("/{workspaceId}/job/{id}")
public R detail(@PathVariable("workspaceId") String workspaceId,
@PathVariable("id") String id) {
JobEntity jobEntity = jobService.queryInfo(id);
return R.ok().putData("record", jobEntity);
}
@GetMapping("/{workspaceId}/job")
public R list(@PathVariable("workspaceId") String workspaceId,
@RequestParam Map<String, Object> params) {
params.put("workspaceId", workspaceId);
Page page = jobService.queryList(params);
return R.ok(page);
}
@PostMapping("/{workspaceId}/job")
public R add(@PathVariable("workspaceId") String workspaceId, String jobCfgId) {
T.VerifyUtil.is(jobCfgId).notNull(RCode.ID_CANNOT_EMPTY);
JobEntity jobEntity = jobService.saveJob(workspaceId, jobCfgId);
return R.ok().putData("record", T.MapUtil.of("id", jobEntity.getId()));
}
@DeleteMapping("/{workspaceId}/job")
public R delete(@PathVariable("workspaceId") String workspaceId,
@RequestParam String ids) {
T.VerifyUtil.is(ids).notEmpty();
List<String> idList = Arrays.asList(ids.split(","));
jobService.removeJob(idList);
return R.ok();
}
@PutMapping("/{workspaceId}/job/cancel")
public R cancel(@PathVariable("workspaceId") String workspaceId,
@RequestParam String ids) {
T.VerifyUtil.is(ids).notEmpty();
List<String> idList = Arrays.asList(ids.split(","));
for (String id : idList) {
cancelJob(id);
}
return R.ok();
}
private void cancelJob(String id) {
JobEntity job = jobService.getById(id);
EnvironmentEntity environment = environmentService.getById(job.getEnvId());
log.info("[cancelJob] [jobId: {}]", id);
JSONObject paramJSONObject = environment.getParamJSONObject();
String url = paramJSONObject.getStr("url");
String token = paramJSONObject.getStr("token");
HttpRequest requestStatus = T.HttpUtil.createGet(String.format("%s/api/v1/env/playbook/%s", url, id));
requestStatus.header("Authorization", token);
if (job.getStatus().contains(JobConstant.JobStatus.RUNNING.getValue())){
while (true){
HttpResponse response = requestStatus.execute();
if (response.isOk()){
break;
}
T.ThreadUtil.sleep(3, TimeUnit.SECONDS);
}
HttpRequest request = T.HttpUtil.createRequest(Method.DELETE, String.format("%s/api/v1/env/playbook/%s", url, id));
request.header("Authorization", token);
HttpResponse response = request.execute();
log.info("[cancelJob] [request env stop playbook] [status: {}]", response.body());
}
if (job.getStatus().contains(JobConstant.JobStatus.PENDING.getValue())){
jobQueueManager.removeJob(job);
}
Thread runningThread = Constants.RUNNING_JOB_THREAD.get(id);
if (runningThread != null) {
runningThread.interrupt();
}
Thread resultThread = Constants.RESULT_JOB_THREAD.get(id);
if (resultThread != null) {
resultThread.interrupt();
}
EnvironmentSessionEntity session = sessionService.getOne(new LambdaQueryWrapper<EnvironmentSessionEntity>()
.eq(EnvironmentSessionEntity::getJobId, id)
.eq(EnvironmentSessionEntity::getStatus, 1));
if (T.ObjectUtil.isNotEmpty(session)) {
environmentService.removeSession(session.getId());
}
// update state
jobService.update(new LambdaUpdateWrapper<JobEntity>()
.eq(JobEntity::getId, id)
.set(JobEntity::getStatus, "cancel")
);
}
@GetMapping("/{workspaceId}/job/{id}/log")
public R getJobLog(@PathVariable("workspaceId") String workspaceId,
@PathVariable("id") String id,
@RequestParam Integer offset) {
offset = offset == null ? 0 : offset;
Map result = jobService.queryJobLog(id, offset);
return R.ok().putData("record", result);
}
@GetMapping("/{workspaceId}/job/cfg/{id}")
public R jobCfgInfo(@PathVariable("workspaceId") String workspaceId,
@PathVariable("id") String id) {
JobCfgEntity entity = jobCfgService.info(id);
return R.ok().putData("record", entity);
}
@GetMapping("/{workspaceId}/job/cfg")
public R jobCfgList(@PathVariable("workspaceId") String workspaceId,
@RequestParam Map<String, Object> params) {
Page page = jobCfgService.queryList(params);
return R.ok(page);
}
@PostMapping("/{workspaceId}/job/cfg")
public R saveCfg(@PathVariable("workspaceId") String workspaceId,
@RequestBody JobCfgEntity cfg) {
VerifyUtil.is(cfg).notNull()
.and(cfg.getName()).notEmpty(RCode.NAME_CANNOT_EMPTY)
.and(cfg.getPlaybookId()).notEmpty(RCode.PLAYBOOK_ID_CANNOT_EMPTY)
.and(cfg.getPackageId()).notEmpty(RCode.PACKAGE_ID_CANNOT_EMPTY)
.and(cfg.getType()).notEmpty(RCode.JOB_CFG_TYPE_CANNOT_EMPTY)
.and(cfg.getStatus()).notEmpty(RCode.JOB_CFG_STATUS_CANNOT_EMPTY);
JobCfgEntity jobCfgEntity = jobCfgService.saveJobCfg(workspaceId, cfg);
return R.ok().putData("record", T.MapUtil.of("id", jobCfgEntity.getId()));
}
@PutMapping("/{workspaceId}/job/cfg")
public R updateCfg(@PathVariable("workspaceId") String workspaceId,
@RequestBody JobCfgEntity cfg) {
VerifyUtil.is(cfg).notNull()
.and(cfg.getId()).notEmpty(RCode.ID_CANNOT_EMPTY)
.and(cfg.getName()).notEmpty(RCode.NAME_CANNOT_EMPTY)
.and(cfg.getPlaybookId()).notEmpty(RCode.PLAYBOOK_ID_CANNOT_EMPTY)
.and(cfg.getPackageId()).notEmpty(RCode.PACKAGE_ID_CANNOT_EMPTY)
.and(cfg.getType()).notEmpty(RCode.JOB_CFG_TYPE_CANNOT_EMPTY)
.and(cfg.getStatus()).notEmpty(RCode.JOB_CFG_STATUS_CANNOT_EMPTY);
JobCfgEntity jobCfgEntity = jobCfgService.updateCfg(workspaceId, cfg);
return R.ok().putData("record", T.MapUtil.of("id", jobCfgEntity.getId()));
}
@DeleteMapping("/{workspaceId}/job/cfg")
public R deleteJobCfg(@PathVariable("workspaceId") String workspaceId,
String ids) {
VerifyUtil.is(ids).notEmpty();
List<String> idList = Arrays.asList(ids.split(","));
jobCfgService.removeBatchByIds(idList);
return R.ok();
}
}

View File

@@ -1,4 +1,4 @@
package net.geedge.asw.module.runner.controller; package net.geedge.asw.module.job.controller;
import cn.hutool.core.date.DatePattern; import cn.hutool.core.date.DatePattern;
import cn.hutool.core.date.DateUtil; import cn.hutool.core.date.DateUtil;
@@ -13,8 +13,8 @@ import jakarta.servlet.http.HttpServletResponse;
import net.geedge.asw.common.config.SpringContextUtils; import net.geedge.asw.common.config.SpringContextUtils;
import net.geedge.asw.common.util.*; import net.geedge.asw.common.util.*;
import net.geedge.asw.module.feign.client.WebSharkClient; import net.geedge.asw.module.feign.client.WebSharkClient;
import net.geedge.asw.module.runner.entity.PcapEntity; import net.geedge.asw.module.job.entity.PcapEntity;
import net.geedge.asw.module.runner.service.IPcapService; import net.geedge.asw.module.job.service.IPcapService;
import org.springframework.beans.factory.annotation.Autowired; import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value; import org.springframework.beans.factory.annotation.Value;
import org.springframework.http.HttpStatus; import org.springframework.http.HttpStatus;
@@ -32,7 +32,7 @@ import java.util.Map;
import java.util.stream.Collectors; import java.util.stream.Collectors;
@RestController @RestController
@RequestMapping("/api/v1/pcap") @RequestMapping("/api/v1/workspace")
public class PcapController { public class PcapController {
private static final Log log = Log.get(); private static final Log log = Log.get();
@@ -43,26 +43,26 @@ public class PcapController {
@Value("${webShark.url:127.0.0.1:8085}") @Value("${webShark.url:127.0.0.1:8085}")
private String websharkurl; private String websharkurl;
@GetMapping("/{id}") @GetMapping("/{workspaceId}/pcap/{id}")
public R detail(@PathVariable("id") String id) { public R detail(@PathVariable("id") String id) {
PcapEntity pcapEntity = pcapService.queryInfo(id); PcapEntity pcapEntity = pcapService.queryInfo(id);
return R.ok().putData("record", pcapEntity); return R.ok().putData("record", pcapEntity);
} }
@GetMapping @GetMapping("/{workspaceId}/pcap")
public R list(@RequestParam Map<String, Object> params) { public R list(@PathVariable("workspaceId") String workspaceId, @RequestParam Map<String, Object> params) {
T.VerifyUtil.is(params).notNull() T.VerifyUtil.is(params).notNull();
.and(T.MapUtil.getStr(params, "workspaceId")).notEmpty(RCode.WORKSPACE_ID_CANNOT_EMPTY);
params.put("workspaceId", workspaceId);
Page page = pcapService.queryList(params); Page page = pcapService.queryList(params);
return R.ok(page); return R.ok(page);
} }
@PostMapping @PostMapping("/{workspaceId}/pcap")
@Transactional(rollbackFor = Exception.class) @Transactional(rollbackFor = Exception.class)
public R add(@RequestParam(value = "files", required = true) List<MultipartFile> fileList, public R add(@RequestParam(value = "files", required = true) List<MultipartFile> fileList,
@RequestParam(value = "descriptions", required = false) List<String> descriptionList, @RequestParam(value = "descriptions", required = false) List<String> descriptionList,
@RequestParam(required = false) String workspaceId) throws IOException { @PathVariable("workspaceId") String workspaceId) throws IOException {
T.VerifyUtil.is(workspaceId).notEmpty(RCode.WORKSPACE_ID_CANNOT_EMPTY); T.VerifyUtil.is(workspaceId).notEmpty(RCode.WORKSPACE_ID_CANNOT_EMPTY);
List<Object> recordList = T.ListUtil.list(true); List<Object> recordList = T.ListUtil.list(true);
@@ -79,9 +79,9 @@ public class PcapController {
return R.ok().putData("records", recordList); return R.ok().putData("records", recordList);
} }
@PutMapping @PutMapping("/{workspaceId}/pcap")
@Transactional(rollbackFor = Exception.class) @Transactional(rollbackFor = Exception.class)
public R update(@RequestBody List<Map<String, String>> body) { public R update(@PathVariable("workspaceId") String workspaceId, @RequestBody List<Map<String, String>> body) {
List<Object> recordList = T.ListUtil.list(true); List<Object> recordList = T.ListUtil.list(true);
for (Map<String, String> map : body) { for (Map<String, String> map : body) {
String id = T.MapUtil.getStr(map, "id", ""); String id = T.MapUtil.getStr(map, "id", "");
@@ -102,7 +102,7 @@ public class PcapController {
return R.ok().putData("records", recordList); return R.ok().putData("records", recordList);
} }
@DeleteMapping @DeleteMapping("/{workspaceId}/pcap")
public R delete(String[] ids) { public R delete(String[] ids) {
T.VerifyUtil.is(ids).notEmpty(); T.VerifyUtil.is(ids).notEmpty();
@@ -110,7 +110,7 @@ public class PcapController {
return R.ok(); return R.ok();
} }
@PutMapping("/parse2session") @PutMapping("/{workspaceId}/pcap/parse2session")
public R parse2session(String[] ids) { public R parse2session(String[] ids) {
T.VerifyUtil.is(ids).notEmpty(); T.VerifyUtil.is(ids).notEmpty();
@@ -131,7 +131,7 @@ public class PcapController {
} }
@GetMapping("/download") @GetMapping("/{workspaceId}/pcap/download")
public void download(HttpServletResponse response, String ids) throws IOException { public void download(HttpServletResponse response, String ids) throws IOException {
T.VerifyUtil.is(ids).notEmpty(); T.VerifyUtil.is(ids).notEmpty();
List<String> pcapIdList = Arrays.asList(ids.split(",")); List<String> pcapIdList = Arrays.asList(ids.split(","));
@@ -150,13 +150,14 @@ public class PcapController {
} }
} }
@GetMapping("/{id}/webshark") @GetMapping("/{workspaceId}/pcap/{id}/webshark")
public R webshark(@PathVariable String id) { public R webshark(@PathVariable("id") String id) {
T.VerifyUtil.is(id).notEmpty(); T.VerifyUtil.is(id).notEmpty();
HashMap<Object, Object> result = T.MapUtil.newHashMap(); HashMap<Object, Object> result = T.MapUtil.newHashMap();
PcapEntity pcap = pcapService.getById(id); PcapEntity pcap = pcapService.getById(id);
File pcapFile = pcap.getCommonPcapFilePath().toFile(); Map summary = T.JSONUtil.toBean(pcap.getSummary(), Map.class);
File pcapFile = FileUtil.file(T.MapUtil.getStr(summary, "commentPath"));
pcapFile = FileUtil.exist(pcapFile) ? pcapFile : T.FileUtil.file(pcap.getPath()); pcapFile = FileUtil.exist(pcapFile) ? pcapFile : T.FileUtil.file(pcap.getPath());
String uploadFileName = T.StrUtil.concat(true, id, ".", T.FileUtil.getSuffix(pcapFile)); String uploadFileName = T.StrUtil.concat(true, id, ".", T.FileUtil.getSuffix(pcapFile));
File newFile = FileUtil.copy(pcapFile, FileUtil.file(Constants.TEMP_PATH, uploadFileName), false); File newFile = FileUtil.copy(pcapFile, FileUtil.file(Constants.TEMP_PATH, uploadFileName), false);
@@ -180,7 +181,7 @@ public class PcapController {
} }
@PutMapping("/unparse2session") @PutMapping("/{workspaceId}/pcap/unparse2session")
public R unparse2session(String[] ids) { public R unparse2session(String[] ids) {
T.VerifyUtil.is(ids).notEmpty(); T.VerifyUtil.is(ids).notEmpty();
@@ -188,14 +189,14 @@ public class PcapController {
return R.ok(); return R.ok();
} }
@GetMapping("/explore") @GetMapping("/{workspaceId}/pcap/explore")
public R explore(@RequestParam String workspaceId, @RequestParam String pcapIds, @RequestParam(required = false) String protocol, @RequestParam(required = false) String streamId) { public R explore(@PathVariable("workspaceId") String workspaceId, @RequestParam String pcapIds, @RequestParam(required = false) String protocol, @RequestParam(required = false) String streamId) {
String discoverUrl = pcapService.generateKibanaDiscoverUrl(workspaceId, pcapIds, protocol, streamId); String discoverUrl = pcapService.generateKibanaDiscoverUrl(workspaceId, pcapIds, protocol, streamId);
return R.ok().putData("url", discoverUrl); return R.ok().putData("url", discoverUrl);
} }
@GetMapping("/dashboard") @GetMapping("/{workspaceId}/pcap/dashboard")
public R dashboard(@RequestParam String workspaceId, @RequestParam String pcapIds) { public R dashboard(@PathVariable("workspaceId") String workspaceId, @RequestParam String pcapIds) {
String dashboardUrl = pcapService.generateKibanaDashboardUrl(workspaceId, pcapIds); String dashboardUrl = pcapService.generateKibanaDashboardUrl(workspaceId, pcapIds);
return R.ok().putData("url", dashboardUrl); return R.ok().putData("url", dashboardUrl);
} }

View File

@@ -1,4 +1,4 @@
package net.geedge.asw.module.runner.controller; package net.geedge.asw.module.job.controller;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page; import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import jakarta.servlet.http.HttpServletResponse; import jakarta.servlet.http.HttpServletResponse;
@@ -6,8 +6,8 @@ import net.geedge.asw.common.util.R;
import net.geedge.asw.common.util.RCode; import net.geedge.asw.common.util.RCode;
import net.geedge.asw.common.util.ResponseUtil; import net.geedge.asw.common.util.ResponseUtil;
import net.geedge.asw.common.util.T; import net.geedge.asw.common.util.T;
import net.geedge.asw.module.runner.entity.PlaybookEntity; import net.geedge.asw.module.job.entity.PlaybookEntity;
import net.geedge.asw.module.runner.service.IPlaybookService; import net.geedge.asw.module.job.service.IPlaybookService;
import org.springframework.beans.factory.annotation.Autowired; import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.MediaType; import org.springframework.http.MediaType;
import org.springframework.web.bind.annotation.*; import org.springframework.web.bind.annotation.*;
@@ -27,7 +27,7 @@ public class PlaybookController {
@GetMapping("/{workspaceId}/playbook/{id}") @GetMapping("/{workspaceId}/playbook/{id}")
public R detail(@PathVariable("workspaceId") String workspaceId, @PathVariable("id") String id) { public R detail(@PathVariable("workspaceId") String workspaceId, @PathVariable("id") String id) {
PlaybookEntity playbook = playbookService.detail(workspaceId, id); PlaybookEntity playbook = playbookService.detail(workspaceId, id);
return R.ok().put("record", playbook); return R.ok().putData("record", playbook);
} }
@GetMapping("/{workspaceId}/playbook") @GetMapping("/{workspaceId}/playbook")
@@ -40,11 +40,21 @@ public class PlaybookController {
public R save(@PathVariable("workspaceId") String workspaceId, public R save(@PathVariable("workspaceId") String workspaceId,
@RequestParam("file") MultipartFile file, @RequestParam("file") MultipartFile file,
@RequestParam("name") String name, @RequestParam("name") String name,
@RequestParam("type") String type,
@RequestParam(value = "description", required = false) String description) { @RequestParam(value = "description", required = false) String description) {
PlaybookEntity playbook = playbookService.savePlaybook(workspaceId, file, name, description); PlaybookEntity playbook = playbookService.savePlaybook(workspaceId, file, name, type, description);
return R.ok().put("record", playbook); return R.ok().put("record", playbook);
} }
@PutMapping("/{workspaceId}/playbook/{playbookId}")
public R update(@PathVariable("workspaceId") String workspaceId,
@PathVariable("playbookId") String playbookId,
@RequestParam("name") String name,
@RequestParam(value = "description", required = false) String description) {
PlaybookEntity playbook = playbookService.updatePlaybook(workspaceId, playbookId, name, description);
return R.ok().putData("record", playbook);
}
@DeleteMapping("/{workspaceId}/playbook") @DeleteMapping("/{workspaceId}/playbook")
public R delete(@PathVariable("workspaceId") String workspaceId, public R delete(@PathVariable("workspaceId") String workspaceId,
@RequestParam("ids") String ids) { @RequestParam("ids") String ids) {
@@ -60,6 +70,6 @@ public class PlaybookController {
T.VerifyUtil.is(entity).notNull(RCode.SYS_RECORD_NOT_FOUND); T.VerifyUtil.is(entity).notNull(RCode.SYS_RECORD_NOT_FOUND);
File playbookFile = T.FileUtil.file(entity.getPath()); File playbookFile = T.FileUtil.file(entity.getPath());
ResponseUtil.downloadFile(response, MediaType.APPLICATION_OCTET_STREAM_VALUE, entity.getName(), T.FileUtil.readBytes(playbookFile)); ResponseUtil.downloadFile(response, MediaType.APPLICATION_OCTET_STREAM_VALUE, playbookFile.getName(), T.FileUtil.readBytes(playbookFile));
} }
} }

View File

@@ -1,16 +1,15 @@
package net.geedge.asw.module.runner.dao; package net.geedge.asw.module.job.dao;
import com.baomidou.mybatisplus.core.mapper.BaseMapper; import com.baomidou.mybatisplus.core.mapper.BaseMapper;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page; import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import net.geedge.asw.module.runner.entity.RunnerEntity; import net.geedge.asw.module.job.entity.JobCfgEntity;
import org.apache.ibatis.annotations.Mapper; import org.apache.ibatis.annotations.Mapper;
import java.util.List; import java.util.List;
import java.util.Map; import java.util.Map;
@Mapper @Mapper
public interface RunnerDao extends BaseMapper<RunnerEntity>{ public interface JobCfgDao extends BaseMapper<JobCfgEntity> {
List<RunnerEntity> queryList(Page page, Map<String, Object> params);
List<JobCfgEntity> queryList(Page page, Map<String, Object> params);
} }

View File

@@ -1,10 +1,9 @@
package net.geedge.asw.module.runner.dao; package net.geedge.asw.module.job.dao;
import com.baomidou.mybatisplus.core.mapper.BaseMapper; import com.baomidou.mybatisplus.core.mapper.BaseMapper;
import com.baomidou.mybatisplus.core.metadata.IPage; import com.baomidou.mybatisplus.core.metadata.IPage;
import net.geedge.asw.module.runner.entity.JobEntity; import net.geedge.asw.module.job.entity.JobEntity;
import org.apache.ibatis.annotations.Mapper; import org.apache.ibatis.annotations.Mapper;
import org.apache.ibatis.annotations.Param;
import java.util.List; import java.util.List;
import java.util.Map; import java.util.Map;

View File

@@ -1,8 +1,8 @@
package net.geedge.asw.module.runner.dao; package net.geedge.asw.module.job.dao;
import com.baomidou.mybatisplus.core.mapper.BaseMapper; import com.baomidou.mybatisplus.core.mapper.BaseMapper;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page; import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import net.geedge.asw.module.runner.entity.PcapEntity; import net.geedge.asw.module.job.entity.PcapEntity;
import org.apache.ibatis.annotations.Mapper; import org.apache.ibatis.annotations.Mapper;
import java.util.List; import java.util.List;

View File

@@ -1,8 +1,8 @@
package net.geedge.asw.module.runner.dao; package net.geedge.asw.module.job.dao;
import com.baomidou.mybatisplus.core.mapper.BaseMapper; import com.baomidou.mybatisplus.core.mapper.BaseMapper;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page; import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import net.geedge.asw.module.runner.entity.PlaybookEntity; import net.geedge.asw.module.job.entity.PlaybookEntity;
import org.apache.ibatis.annotations.Mapper; import org.apache.ibatis.annotations.Mapper;
import org.apache.ibatis.annotations.Param; import org.apache.ibatis.annotations.Param;

View File

@@ -0,0 +1,63 @@
package net.geedge.asw.module.job.entity;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableField;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
import com.fasterxml.jackson.annotation.JsonProperty;
import lombok.Data;
import net.geedge.asw.module.app.entity.PackageEntity;
import net.geedge.asw.module.environment.entity.EnvironmentEntity;
import net.geedge.asw.module.sys.entity.SysUserEntity;
import net.geedge.asw.module.workspace.entity.WorkspaceEntity;
import java.util.List;
@Data
@TableName("job_cfg")
public class JobCfgEntity {
@TableId(type = IdType.ASSIGN_UUID)
private String id;
private String name;
private String description;
private String packageId;
private String envId;
private String playbookId;
private String type;
private String cron;
private String status;
private String parameters;
private Long createTimestamp;
private Long updateTimestamp;
private String createUserId;
private String updateUserId;
private String workspaceId;
@TableField(exist = false)
@JsonProperty(value = "package")
private PackageEntity pkg;
@TableField(exist = false)
private EnvironmentEntity environment;
@TableField(exist = false)
private PlaybookEntity playbook;
@TableField(exist = false)
private List<PcapEntity> pcap;
@TableField(exist = false)
private SysUserEntity createUser;
@TableField(exist = false)
private SysUserEntity updateUser;
@TableField(exist = false)
private WorkspaceEntity workspace;
@TableField(exist = false)
private JobEntity lastJobResult;
}

View File

@@ -1,13 +1,19 @@
package net.geedge.asw.module.runner.entity; package net.geedge.asw.module.job.entity;
import com.baomidou.mybatisplus.annotation.IdType; import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableField; import com.baomidou.mybatisplus.annotation.TableField;
import com.baomidou.mybatisplus.annotation.TableId; import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName; import com.baomidou.mybatisplus.annotation.TableName;
import com.fasterxml.jackson.annotation.JsonIgnore;
import com.fasterxml.jackson.annotation.JsonProperty; import com.fasterxml.jackson.annotation.JsonProperty;
import lombok.Data; import lombok.Data;
import net.geedge.asw.module.app.entity.PackageEntity; import net.geedge.asw.module.app.entity.PackageEntity;
import net.geedge.asw.module.environment.entity.EnvironmentEntity; import net.geedge.asw.module.environment.entity.EnvironmentEntity;
import net.geedge.asw.module.environment.entity.EnvironmentSessionEntity;
import net.geedge.asw.module.sys.entity.SysUserEntity;
import net.geedge.asw.module.workspace.entity.WorkspaceEntity;
import java.util.List;
@Data @Data
@TableName("job") @TableName("job")
@@ -15,15 +21,18 @@ public class JobEntity {
@TableId(type = IdType.ASSIGN_UUID) @TableId(type = IdType.ASSIGN_UUID)
private String id; private String id;
private String jobCfgId;
private String packageId; private String packageId;
private String envId; private String envId;
private String playbookId; private String playbookId;
private String playbookParam; private String parameters;
private Long startTimestamp; private Long startTimestamp;
private Long endTimestamp; private Long endTimestamp;
private String status; private String status;
private String pcapId; @JsonIgnore
private String logPath; private String logPath;
@JsonIgnore
private String artifactsPath;
private Long createTimestamp; private Long createTimestamp;
private Long updateTimestamp; private Long updateTimestamp;
@@ -45,4 +54,18 @@ public class JobEntity {
@TableField(exist = false) @TableField(exist = false)
private PlaybookEntity playbook; private PlaybookEntity playbook;
@TableField(exist = false)
private List<PcapEntity> pcap;
@TableField(exist = false)
private SysUserEntity createUser;
@TableField(exist = false)
private WorkspaceEntity workspace;
@TableField(exist = false)
private EnvironmentSessionEntity session;
@TableField(exist = false)
private JobCfgEntity jobCfg;
} }

View File

@@ -1,4 +1,4 @@
package net.geedge.asw.module.runner.entity; package net.geedge.asw.module.job.entity;
import com.baomidou.mybatisplus.annotation.IdType; import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableField; import com.baomidou.mybatisplus.annotation.TableField;
@@ -7,13 +7,15 @@ import com.baomidou.mybatisplus.annotation.TableName;
import com.fasterxml.jackson.annotation.JsonIgnore; import com.fasterxml.jackson.annotation.JsonIgnore;
import com.fasterxml.jackson.annotation.JsonProperty; import com.fasterxml.jackson.annotation.JsonProperty;
import lombok.Data; import lombok.Data;
import net.geedge.asw.common.util.T; import net.geedge.asw.common.util.Constants;
import net.geedge.asw.common.util.FileResourceUtil;
import net.geedge.asw.module.app.entity.ApplicationEntity; import net.geedge.asw.module.app.entity.ApplicationEntity;
import net.geedge.asw.module.app.entity.PackageEntity; import net.geedge.asw.module.app.entity.PackageEntity;
import net.geedge.asw.module.environment.entity.EnvironmentEntity; import net.geedge.asw.module.environment.entity.EnvironmentEntity;
import net.geedge.asw.module.sys.entity.SysUserEntity;
import net.geedge.asw.module.workspace.entity.WorkspaceEntity; import net.geedge.asw.module.workspace.entity.WorkspaceEntity;
import java.nio.file.Path; import java.io.File;
@Data @Data
@TableName("pcap") @TableName("pcap")
@@ -23,6 +25,7 @@ public class PcapEntity {
private String id; private String id;
private String name; private String name;
private String description; private String description;
@JsonIgnore
private String path; private String path;
private Long size; private Long size;
private String md5; private String md5;
@@ -35,7 +38,6 @@ public class PcapEntity {
@TableField(exist = false) @TableField(exist = false)
private WorkspaceEntity workspace; private WorkspaceEntity workspace;
@TableField(exist = false)
private String jobId; private String jobId;
@TableField(exist = false) @TableField(exist = false)
@@ -51,9 +53,12 @@ public class PcapEntity {
@TableField(exist = false) @TableField(exist = false)
private PlaybookEntity playbook; private PlaybookEntity playbook;
@TableField(exist = false)
private SysUserEntity createUser;
@JsonIgnore @JsonIgnore
public Path getCommonPcapFilePath() { public File getCommonPcapFilePath(String resources) {
return Path.of(T.WebPathUtil.getRootPath(), this.workspaceId, "pcap_comment", this.id + ".pcapng"); return FileResourceUtil.createFile(resources, this.workspaceId, Constants.FileTypeEnum.PCAP.getType(), this.id, this.id + "-comment.pcapng");
} }
} }

View File

@@ -1,9 +1,10 @@
package net.geedge.asw.module.runner.entity; package net.geedge.asw.module.job.entity;
import com.baomidou.mybatisplus.annotation.IdType; import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableField; import com.baomidou.mybatisplus.annotation.TableField;
import com.baomidou.mybatisplus.annotation.TableId; import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName; import com.baomidou.mybatisplus.annotation.TableName;
import com.fasterxml.jackson.annotation.JsonIgnore;
import lombok.Data; import lombok.Data;
import net.geedge.asw.module.sys.entity.SysUserEntity; import net.geedge.asw.module.sys.entity.SysUserEntity;
@@ -14,7 +15,9 @@ public class PlaybookEntity {
@TableId(type = IdType.ASSIGN_UUID) @TableId(type = IdType.ASSIGN_UUID)
private String id; private String id;
private String name; private String name;
private String type;
private String tags; private String tags;
@JsonIgnore
private String path; private String path;
private String description; private String description;

View File

@@ -0,0 +1,337 @@
package net.geedge.asw.module.job.job;
import cn.hutool.core.net.url.UrlBuilder;
import cn.hutool.http.HttpRequest;
import cn.hutool.http.HttpResponse;
import cn.hutool.json.JSONArray;
import cn.hutool.json.JSONObject;
import cn.hutool.log.Log;
import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
import net.geedge.asw.common.util.Constants;
import net.geedge.asw.common.util.FileResourceUtil;
import net.geedge.asw.common.util.RCode;
import net.geedge.asw.common.util.T;
import net.geedge.asw.module.app.entity.PackageEntity;
import net.geedge.asw.module.app.service.IPackageService;
import net.geedge.asw.module.environment.entity.EnvironmentEntity;
import net.geedge.asw.module.environment.entity.EnvironmentSessionEntity;
import net.geedge.asw.module.environment.service.IEnvironmentService;
import net.geedge.asw.module.environment.service.IEnvironmentSessionService;
import net.geedge.asw.module.job.entity.JobEntity;
import net.geedge.asw.module.job.entity.PcapEntity;
import net.geedge.asw.module.job.service.IJobService;
import net.geedge.asw.module.job.service.IPcapService;
import net.geedge.asw.module.job.util.JobConstant;
import net.lingala.zip4j.ZipFile;
import net.lingala.zip4j.model.FileHeader;
import net.lingala.zip4j.model.ZipParameters;
import net.lingala.zip4j.model.enums.CompressionLevel;
import net.lingala.zip4j.model.enums.CompressionMethod;
import org.apache.commons.lang3.time.StopWatch;
import org.quartz.DisallowConcurrentExecution;
import org.quartz.JobExecutionContext;
import org.quartz.JobExecutionException;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.scheduling.quartz.QuartzJobBean;
import org.springframework.transaction.annotation.Transactional;
import java.io.*;
import java.util.List;
@DisallowConcurrentExecution
@SuppressWarnings("all")
public class JobPlaybookExecResultChecker extends QuartzJobBean {
private static final Log log = Log.get();
@Autowired
private IEnvironmentService environmentService;
@Autowired
private IPcapService pcapService;
@Autowired
private IJobService jobService;
@Autowired
private IPackageService packageService;
@Autowired
private IEnvironmentSessionService environmentSessionService;
@Value("${asw.resources.path:resources}")
private String resources;
@Override
protected void executeInternal(JobExecutionContext context) throws JobExecutionException {
Thread.currentThread().setName("JobPlaybookExecResultChecker");
log.info("[JobPlaybookExecResultChecker] [begin]");
StopWatch sw = new StopWatch();
sw.start();
try {
this.playbookExecResultChecker();
} catch (Exception e) {
log.error(e, "[JobPlaybookExecResultChecker] [error]");
} finally {
sw.stop();
}
log.info("[JobPlaybookExecResultChecker] [finshed] [Run Time: {}]", sw.toString());
}
private void playbookExecResultChecker() {
List<JobEntity> jobList = jobService.list(new LambdaQueryWrapper<JobEntity>().eq(JobEntity::getStatus, JobConstant.JobStatus.RUNNING.getValue()));
if (jobList.isEmpty()) {
return;
}
for (JobEntity job : jobList) {
Thread.ofVirtual().start(() -> {
String id = job.getId();
EnvironmentEntity environment = environmentService.getById(job.getEnvId());
log.info("[playbookExecResultChecker] [jobId: {}]", id);
JSONObject paramJSONObject = environment.getParamJSONObject();
String url = paramJSONObject.getStr("url");
String token = paramJSONObject.getStr("token");
HttpRequest request = T.HttpUtil.createGet(String.format("%s/api/v1/env/playbook/%s", url, id));
request.header("Authorization", token);
HttpResponse response = request.execute();
log.info("[playbookExecResultChecker] [request env api] [env: {}] [status: {}]", environment.getId(), response.getStatus());
if (response.isOk()) {
String result = response.body();
if (log.isDebugEnabled()) {
log.debug("[playbookExecResultChecker] [env: {}] [result: {}]", environment.getId(), result);
}
JSONObject jsonObject = T.JSONUtil.parseObj(result);
if (T.ObjectUtil.equal(RCode.SUCCESS.getCode(), jsonObject.getInt("code"))) {
JSONObject data = jsonObject.getJSONObject("data");
String status = data.getStr("status");
switch (status) {
case "running":
Constants.RUNNING_JOB_THREAD.computeIfAbsent(id, jobId -> startGetJobLogThread(job, environment));
break;
case "error":
Constants.RESULT_JOB_THREAD.computeIfAbsent(id, jobId -> startGetJobResultThread(job, environment, JobConstant.JobStatus.FAILED.getValue()));
break;
case "done":
Constants.RESULT_JOB_THREAD.computeIfAbsent(id, jobId -> startGetJobResultThread(job, environment, JobConstant.JobStatus.PASSED.getValue()));
break;
}
}
}
});
}
}
private Thread startGetJobLogThread(JobEntity job, EnvironmentEntity environment) {
Thread thread = Thread.ofVirtual().start(() -> {
log.info("[playbookExecResultChecker] [startGetJobLogThread] [begin] [job id: {}]", job.getId());
try {
while (true) {
if (isJobInRunningStatus(job)) {
Constants.RUNNING_JOB_THREAD.remove(job.getId());
log.info("[playbookExecResultChecker] [startGetJobLogThread] [finshed ] [job id: {}]", job.getId());
break;
}
performJobLogic(job, environment);
Thread.sleep(2000); // 每 2 秒执行一次
}
} catch (InterruptedException e) {
log.info("[playbookExecResultChecker] [startGetJobLogThread] [stop thread] [job id: {}]", job.getId());
Constants.RUNNING_JOB_THREAD.remove(job.getId());
Thread.currentThread().interrupt(); // 恢复中断状态
}
});
return thread;
}
// 检查 Job 的状态是否为 running
private boolean isJobInRunningStatus(JobEntity jobEntity) {
JobEntity job = jobService.getById(jobEntity.getId());
return job != null && !T.StrUtil.equalsIgnoreCase(job.getStatus(), JobConstant.JobStatus.RUNNING.getValue());
}
/**
* 获取 playbook 执行日志
*
* @param job
* @param environment
*/
private void performJobLogic(JobEntity job, EnvironmentEntity environment) {
File logFile = T.FileUtil.file(job.getLogPath());
Integer offset = 0;
if (logFile.exists()) {
offset = T.FileUtil.readBytes(logFile).length;
}
JSONObject paramJSONObject = environment.getParamJSONObject();
String url = paramJSONObject.getStr("url");
String token = paramJSONObject.getStr("token");
HttpRequest request = T.HttpUtil.createGet(String.format("%s/api/v1/env/playbook/%s/log", url, job.getId()));
request.form("offset", offset);
request.header("Authorization", token);
HttpResponse response = request.execute();
if (response.isOk()) {
String result = response.body();
if (log.isDebugEnabled()) {
log.debug("[playbookExecResultChecker] [performJobLogic] [env: {}] [result: {}]", environment.getId(), result);
}
JSONObject jsonObject = T.JSONUtil.parseObj(result);
if (T.ObjectUtil.equal(RCode.SUCCESS.getCode(), jsonObject.getInt("code"))) {
JSONObject data = jsonObject.getJSONObject("data");
String content = data.getStr("content");
content = T.StrUtil.nullToDefault(content, T.StrUtil.EMPTY);
T.FileUtil.appendString(content, logFile, "UTF-8");
}
}
}
/**
* get pcap log
*
* @param job
* @param value job status: error and done
* @param environment
*/
@Transactional(rollbackFor = Exception.class)
private Thread startGetJobResultThread(JobEntity job, EnvironmentEntity environment, String status) {
Thread thread = Thread.ofVirtual().start(() -> {
File destination = null;
File artifactZip = null;
InputStream inputStream = null;
ZipFile zipFile = null;
ZipFile artifactsZipFile = null;
List<File> artifactFiles = T.ListUtil.list(false);
try {
log.info("[playbookExecResultChecker] [startGetJobResultThread] [job status: {}] [jod id: {}] [time: {}]", status, job.getId(), System.currentTimeMillis());
JSONObject paramJSONObject = environment.getParamJSONObject();
String url = paramJSONObject.getStr("url");
String token = paramJSONObject.getStr("token");
UrlBuilder urlBuilder = UrlBuilder.of(String.format("%s/api/v1/env/playbook/%s/artifact", url, job.getId()));
String parameters = job.getParameters();
if (T.StrUtil.isNotEmpty(parameters)){
JSONObject obj = T.JSONUtil.parseObj(parameters);
JSONArray artifacts = obj.getJSONArray("artifacts");
if (artifacts != null){
for (Object artifact : artifacts) {
urlBuilder.addQuery("artifacts", artifact);
}
}
}
HttpRequest request = T.HttpUtil.createGet(urlBuilder.build());
request.header("Authorization", token);
HttpResponse response = request.execute();
log.info("[playbookExecResultChecker] [startGetJobResultThread] [request env playbook result api] [status: {}] [time: {}]", response.getStatus(), System.currentTimeMillis());
if (response.isOk()) {
destination = T.FileUtil.file(Constants.TEMP_PATH, T.StrUtil.concat(true, job.getId(), ".zip"));
T.FileUtil.writeBytes(response.bodyBytes(), destination);
zipFile = new ZipFile(destination);
List<FileHeader> fileHeaders = zipFile.getFileHeaders();
for (FileHeader fileHeader : fileHeaders) {
// 检查中断状态
if (Thread.currentThread().isInterrupted()) {
log.info("[playbookExecResultChecker] [startGetJobResultThread] [stop thread] [job id: {}]", job.getId());
return; // 中断线程,退出循环
}
// 处理 pcap 文件
if (fileHeader.getFileName().endsWith("pcap")) {
PackageEntity packageEntity = packageService.getById(job.getPackageId());
String fileName = T.StrUtil.concat(true, "job", T.StrUtil.DASHED, T.StrUtil.sub(job.getId(), 0, 8), T.StrUtil.DASHED, packageEntity.getName(), ".pcap");
if (fileHeader.getFileName().contains("all")) {
fileName = T.StrUtil.concat(true, "job", T.StrUtil.DASHED, T.StrUtil.sub(job.getId(), 0, 8), ".pcap");
}
String pcapId = T.StrUtil.uuid();
// 上传 pcap 文件流
File pcapFile = FileResourceUtil.createFile(resources, job.getWorkspaceId(), Constants.FileTypeEnum.PCAP.getType(), pcapId, pcapId + ".pcap");
File parentDir = pcapFile.getParentFile();
if (!parentDir.exists()) {
parentDir.mkdirs();
}
inputStream = zipFile.getInputStream(fileHeader);
T.FileUtil.writeFromStream(inputStream, pcapFile);
PcapEntity entity = new PcapEntity();
entity.setId(pcapId);
entity.setName(fileName);
entity.setSize(pcapFile.length());
entity.setStatus(JobConstant.PcapStatus.UPLOADED.getValue());
entity.setCreateTimestamp(System.currentTimeMillis());
entity.setCreateUserId(job.getCreateUserId());
entity.setWorkspaceId(job.getWorkspaceId());
entity.setJobId(job.getId());
entity.setPath(pcapFile.getPath());
// md5
String md5Hex = T.DigestUtil.md5Hex(pcapFile);
entity.setMd5(md5Hex);
pcapService.save(entity);
log.info("[playbookExecResultChecker] [startGetJobResultThread] [upload pcap: {}] [job id: {}] [time: {}]", T.JSONUtil.toJsonStr(entity), job.getId(), System.currentTimeMillis());
} else if (fileHeader.getFileName().equals("result.log")) {
// 处理 log 文件
File logFile = T.FileUtil.file(job.getLogPath());
inputStream = zipFile.getInputStream(fileHeader);
T.FileUtil.writeFromStream(inputStream, logFile);
} else {
File artifactFile = FileResourceUtil.createFile(resources, job.getWorkspaceId(), Constants.FileTypeEnum.JOB.getType(), job.getId(), fileHeader.getFileName());
inputStream = zipFile.getInputStream(fileHeader);
T.FileUtil.writeFromStream(inputStream, artifactFile);
artifactFiles.add(artifactFile);
}
}
if (T.CollUtil.isNotEmpty(artifactFiles) && artifactFiles.size() > 1) {
artifactZip = FileResourceUtil.createFile(resources, job.getWorkspaceId(), Constants.FileTypeEnum.JOB.getType(), job.getId(), T.StrUtil.concat(true, job.getId(), ".zip"));
artifactsZipFile = new ZipFile(artifactZip);
ZipParameters zipParameters = new ZipParameters();
zipParameters.setCompressionMethod(CompressionMethod.DEFLATE); // 压缩方法
zipParameters.setCompressionLevel(CompressionLevel.FASTEST); // 压缩级别,选项有 FASTEST、ULTRA 等
artifactsZipFile.addFiles(artifactFiles, zipParameters);
job.setArtifactsPath(artifactZip.getPath());
}
if (T.CollUtil.isNotEmpty(artifactFiles) && artifactFiles.size() == 1) {
job.setArtifactsPath(artifactFiles.getFirst().getPath());
}
// update job status
job.setStatus(status);
job.setEndTimestamp(System.currentTimeMillis());
jobService.updateById(job);
// remove session
EnvironmentSessionEntity session = environmentSessionService.getOne(new LambdaQueryWrapper<EnvironmentSessionEntity>()
.eq(EnvironmentSessionEntity::getJobId, job.getId())
.eq(EnvironmentSessionEntity::getStatus, 1)
);
if (session != null) {
environmentService.removeSession(session.getId());
}
log.info("[playbookExecResultChecker] [startGetJobResultThread] [finshed] [job id: {}]", job.getId());
}
} catch (Exception e) {
if (e.getMessage().contains("Closed by interrupt")) {
Thread.currentThread().interrupt(); // 恢复中断状态
}
log.error("[playbookExecResultChecker] [startGetJobResultThread] [error]", e);
} finally {
T.IoUtil.close(zipFile);
T.IoUtil.close(artifactsZipFile);
T.FileUtil.del(destination);
T.IoUtil.close(inputStream);
if (artifactFiles.size() > 1){
artifactFiles.stream().forEach(file-> T.FileUtil.del(file));
}
Constants.RESULT_JOB_THREAD.remove(job.getId());
}
});
return thread;
}
}

View File

@@ -0,0 +1,228 @@
package net.geedge.asw.module.job.job;
import cn.hutool.http.HttpRequest;
import cn.hutool.http.HttpResponse;
import cn.hutool.json.JSONObject;
import cn.hutool.log.Log;
import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
import com.baomidou.mybatisplus.core.conditions.update.LambdaUpdateWrapper;
import net.geedge.asw.common.util.Constants;
import net.geedge.asw.common.util.T;
import net.geedge.asw.module.app.entity.PackageEntity;
import net.geedge.asw.module.app.service.IPackageService;
import net.geedge.asw.module.environment.entity.EnvironmentEntity;
import net.geedge.asw.module.environment.entity.EnvironmentSessionEntity;
import net.geedge.asw.module.environment.service.IEnvironmentService;
import net.geedge.asw.module.environment.service.IEnvironmentSessionService;
import net.geedge.asw.module.job.entity.JobEntity;
import net.geedge.asw.module.job.entity.PlaybookEntity;
import net.geedge.asw.module.job.service.IJobService;
import net.geedge.asw.module.job.service.IPlaybookService;
import net.geedge.asw.module.job.util.JobQueueManager;
import net.geedge.asw.module.job.util.JobConstant;
import org.apache.commons.lang3.time.StopWatch;
import org.quartz.DisallowConcurrentExecution;
import org.quartz.JobExecutionContext;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.scheduling.quartz.QuartzJobBean;
import org.springframework.transaction.annotation.Transactional;
import java.io.File;
import java.util.List;
import java.util.Map;
@DisallowConcurrentExecution
@SuppressWarnings("all")
public class JobPlaybookExecutor extends QuartzJobBean {
private static final Log log = Log.get();
@Autowired
private IJobService jobService;
@Autowired
private IEnvironmentService environmentService;
@Autowired
private IPackageService packageService;
@Autowired
private IPlaybookService playbookService;
@Autowired
private IEnvironmentSessionService environmentSessionService;
@Autowired
private JobQueueManager jobQueueManager;
@Override
protected void executeInternal(JobExecutionContext context) {
Thread.currentThread().setName("JobPlaybookExecutor");
log.info("[JobPlaybookExecutor] [begin]");
StopWatch sw = new StopWatch();
sw.start();
try {
this.playbookExecutor();
} catch (Exception e) {
log.error(e, "[JobPlaybookExecutor] [error]");
} finally {
sw.stop();
}
log.info("[JobPlaybookExecutor] [finshed] [Run Time: {}]", sw.toString());
}
@Transactional(rollbackFor = Exception.class)
public void playbookExecutor() {
List<JobEntity> createdJobs = jobService.list(
new LambdaQueryWrapper<JobEntity>()
.eq(JobEntity::getStatus, JobConstant.JobStatus.CREATED.getValue())
.orderByAsc(JobEntity::getCreateTimestamp)
);
if (T.CollUtil.isNotEmpty(createdJobs)) {
log.info("[JobPlaybookExecutor] [playbookExecutor] [fetching created jobs] [size: {}]", createdJobs.size());
// 将 CREATED 任务加入队列
createdJobs.forEach(jobQueueManager::addJob);
// 更新 createdJobs 状态为 pending
createdJobs.forEach(x -> x.setStatus(JobConstant.JobStatus.PENDING.getValue()));
jobService.updateBatchById(createdJobs);
}
// 处理队列中的任务
if (!jobQueueManager.isAllQueuesEmpty()) {
List<JobEntity> nextJobList = jobQueueManager.fetchNextJob();
for (JobEntity nextJob : nextJobList) {
String envId = nextJob.getEnvId();
log.info("[JobPlaybookExecutor] [playbookExecutor] [Processing jobId: {}] [envId: {}]", nextJob.getId(), envId);
EnvironmentEntity environment = environmentService.getById(envId);
if (!environment.getStatus().equals(1)) {
log.warn("[JobPlaybookExecutor] [playbookExecutor] [environment is not available] [jobId: {}] [envId: {}]", nextJob.getId(), environment.getId());
jobQueueManager.requeueJob(nextJob); // 将任务放回队列
continue;
}
List<EnvironmentSessionEntity> sessionList = environmentSessionService.list(new LambdaQueryWrapper<EnvironmentSessionEntity>()
.eq(EnvironmentSessionEntity::getStatus, "1")
.eq(EnvironmentSessionEntity::getEnvId, envId));
if (T.CollUtil.isNotEmpty(sessionList)) {
log.warn("[JobPlaybookExecutor] [playbookExecutor] [environment is in used] [jobId: {}] [envId: {}]", nextJob.getId(), environment.getId());
jobQueueManager.requeueJob(nextJob); // 将任务放回队列
continue;
}
// update job status running
jobService.update(new LambdaUpdateWrapper<JobEntity>()
.set(JobEntity::getStatus, JobConstant.JobStatus.RUNNING.getValue())
.set(JobEntity::getStartTimestamp, System.currentTimeMillis())
.eq(JobEntity::getId, nextJob.getId())
);
// add session
EnvironmentSessionEntity session = new EnvironmentSessionEntity();
session.setEnvId(envId);
session.setJobId(nextJob.getId());
session.setStatus(1);
session.setUserId("system");
session.setWorkspaceId(nextJob.getWorkspaceId());
session.setStartTimestamp(System.currentTimeMillis());
environmentSessionService.save(session);
// 执行任务
processJobAsync(nextJob, environment, session);
}
}
}
private void processJobAsync(JobEntity job, EnvironmentEntity environment, EnvironmentSessionEntity session) {
T.ThreadUtil.execAsync(() -> {
log.info("[JobPlaybookExecutor] [processJobAsync] [start jobId: {}]", job.getId());
try {
// 执行请求
HttpResponse response = requestEnvironment(job, environment);
if (!response.isOk()) {
String result = response.body();
log.warn("[JobPlaybookExecutor] [processJobAsync] [envId: {}] [result: {}]", environment.getId(), result);
File logFile = T.FileUtil.file(job.getLogPath());
T.FileUtil.appendString(String.format("ERROR: Request %s environment error! msg: %s.\n", environment.getName(), result), logFile, "UTF-8");
// update job status, starTime, updateTimestamp
jobService.update(new LambdaUpdateWrapper<JobEntity>()
.set(JobEntity::getStatus, JobConstant.JobStatus.FAILED.getValue())
.set(JobEntity::getEndTimestamp, System.currentTimeMillis())
.eq(JobEntity::getId, job.getId()));
// remove session
environmentService.removeSession(session.getId());
}
log.info("[JobPlaybookExecutor] [processJobAsync] [Finished jobId: {}]", job.getId());
} catch (Exception e) {
// update job status, starTime, updateTimestamp
jobService.update(new LambdaUpdateWrapper<JobEntity>()
.set(JobEntity::getStatus, JobConstant.JobStatus.FAILED.getValue())
.set(JobEntity::getEndTimestamp, System.currentTimeMillis())
.eq(JobEntity::getId, job.getId()));
// remove session
environmentService.removeSession(session.getId());
throw new RuntimeException(e);
}
});
}
private HttpResponse requestEnvironment(JobEntity job, EnvironmentEntity environment) {
File zipFile = null;
try {
String playbookId = job.getPlaybookId();
String packageId = job.getPackageId();
// package playbook file
PackageEntity packageEntity = packageService.getById(packageId);
File packageFile = T.FileUtil.file(packageEntity.getPath());
PlaybookEntity playbook = playbookService.getById(playbookId);
File playbookFile = T.FileUtil.file(playbook.getPath());
// zip
zipFile = T.FileUtil.file(Constants.TEMP_PATH, T.StrUtil.concat(true, job.getId(), ".zip"));
T.ZipUtil.zip(zipFile, true, packageFile, playbookFile);
JSONObject paramJSONObject = environment.getParamJSONObject();
String url = paramJSONObject.getStr("url");
String token = paramJSONObject.getStr("token");
// parameters
String packageName = packageEntity.getIdentifier();
String parameters = job.getParameters();
Map<String, Object> params = T.MapUtil.newHashMap();
if (T.StrUtil.isNotEmpty(parameters)) {
params = T.JSONUtil.toBean(parameters, Map.class);
} else {
params.put("reInstall", true);
params.put("clearCache", true);
params.put("unInstall", true);
}
// build request
log.info("[JobPlaybookExecutor] [requestEnvironment] [jobId: {}] [envId: {}] [playbookId: {}] [packageId: {}]", job.getId(), environment.getId(), playbookId, packageId);
HttpRequest request = T.HttpUtil.createPost(String.format("%s/api/v1/env/playbook", url));
request.header("Authorization", token);
request.form("file", zipFile);
request.form("id", job.getId());
request.form("packageName", packageName);
request.form("type", playbook.getType());
for (Map.Entry<String, Object> param : params.entrySet()) {
request.form(param.getKey(), param.getValue());
}
HttpResponse response = request.execute();
return response;
} catch (Exception e) {
log.error("[JobPlaybookExecutor] [requestEnvironment] [error] [jobId: {}]", job.getId(), e);
throw new RuntimeException(e);
} finally {
T.FileUtil.del(zipFile);
}
}
}

View File

@@ -0,0 +1,18 @@
package net.geedge.asw.module.job.service;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import com.baomidou.mybatisplus.extension.service.IService;
import net.geedge.asw.module.job.entity.JobCfgEntity;
import java.util.Map;
public interface IJobCfgService extends IService<JobCfgEntity> {
JobCfgEntity info(String id);
Page queryList(Map<String, Object> params);
JobCfgEntity saveJobCfg(String workspaceId, JobCfgEntity cfg);
JobCfgEntity updateCfg(String workspaceId, JobCfgEntity cfg);
}

View File

@@ -0,0 +1,21 @@
package net.geedge.asw.module.job.service;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import com.baomidou.mybatisplus.extension.service.IService;
import net.geedge.asw.module.job.entity.JobEntity;
import java.util.List;
import java.util.Map;
public interface IJobService extends IService<JobEntity>{
JobEntity queryInfo(String id);
Page queryList(Map<String, Object> params);
JobEntity saveJob(String workspaceId, String jobCfgId);
void removeJob(List<String> ids);
Map queryJobLog(String id, Integer offset);
}

View File

@@ -1,8 +1,8 @@
package net.geedge.asw.module.runner.service; package net.geedge.asw.module.job.service;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page; import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import com.baomidou.mybatisplus.extension.service.IService; import com.baomidou.mybatisplus.extension.service.IService;
import net.geedge.asw.module.runner.entity.PcapEntity; import net.geedge.asw.module.job.entity.PcapEntity;
import org.springframework.core.io.Resource; import org.springframework.core.io.Resource;
import java.util.Map; import java.util.Map;

View File

@@ -1,8 +1,8 @@
package net.geedge.asw.module.runner.service; package net.geedge.asw.module.job.service;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page; import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import com.baomidou.mybatisplus.extension.service.IService; import com.baomidou.mybatisplus.extension.service.IService;
import net.geedge.asw.module.runner.entity.PlaybookEntity; import net.geedge.asw.module.job.entity.PlaybookEntity;
import org.springframework.web.multipart.MultipartFile; import org.springframework.web.multipart.MultipartFile;
import java.util.Map; import java.util.Map;
@@ -13,7 +13,9 @@ public interface IPlaybookService extends IService<PlaybookEntity>{
Page queryList(String workspaceId, Map params); Page queryList(String workspaceId, Map params);
PlaybookEntity savePlaybook(String workspaceId, MultipartFile file, String name, String description); PlaybookEntity savePlaybook(String workspaceId, MultipartFile file, String name, String type, String description);
void delete(String workspaceId, String ids); void delete(String workspaceId, String ids);
PlaybookEntity updatePlaybook(String workspaceId, String playbookId, String name, String description);
} }

View File

@@ -0,0 +1,159 @@
package net.geedge.asw.module.job.service.impl;
import cn.dev33.satoken.stp.StpUtil;
import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;
import net.geedge.asw.common.config.Query;
import net.geedge.asw.common.util.*;
import net.geedge.asw.module.app.entity.PackageEntity;
import net.geedge.asw.module.app.service.IPackageService;
import net.geedge.asw.module.environment.entity.EnvironmentEntity;
import net.geedge.asw.module.environment.service.IEnvironmentService;
import net.geedge.asw.module.job.dao.JobCfgDao;
import net.geedge.asw.module.job.entity.JobCfgEntity;
import net.geedge.asw.module.job.entity.JobEntity;
import net.geedge.asw.module.job.entity.PcapEntity;
import net.geedge.asw.module.job.entity.PlaybookEntity;
import net.geedge.asw.module.job.service.IJobCfgService;
import net.geedge.asw.module.job.service.IJobService;
import net.geedge.asw.module.job.service.IPcapService;
import net.geedge.asw.module.job.service.IPlaybookService;
import net.geedge.asw.module.sys.entity.SysUserEntity;
import net.geedge.asw.module.sys.service.ISysUserService;
import net.geedge.asw.module.workspace.entity.WorkspaceEntity;
import net.geedge.asw.module.workspace.service.IWorkspaceService;
import org.quartz.CronExpression;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import java.util.List;
import java.util.Map;
@Service
public class JobCfgServiceImpl extends ServiceImpl<JobCfgDao, JobCfgEntity> implements IJobCfgService {
@Autowired
private IPackageService packageService;
@Autowired
private IPlaybookService playbookService;
@Autowired
private IEnvironmentService environmentService;
@Autowired
private IJobService jobService;
@Autowired
private IPcapService pcapService;
@Autowired
private ISysUserService userService;
@Autowired
private IWorkspaceService workspaceService;
@Override
public JobCfgEntity info(String id) {
JobCfgEntity cfg = this.getById(id);
PackageEntity pkg = packageService.getById(cfg.getPackageId());
if (T.ObjectUtil.isNotEmpty(pkg)) {
cfg.setPkg(pkg);
}
EnvironmentEntity environment = environmentService.getById(cfg.getEnvId());
if (T.ObjectUtil.isNotEmpty(environment)) {
cfg.setEnvironment(environment);
}
PlaybookEntity playbook = playbookService.getById(cfg.getPlaybookId());
if (T.ObjectUtil.isNotEmpty(playbook)) {
cfg.setPlaybook(playbook);
}
SysUserEntity createUser = userService.getById(cfg.getCreateUserId());
if (T.ObjectUtil.isNotEmpty(createUser)) {
cfg.setCreateUser(createUser);
}
SysUserEntity updateUser = userService.getById(cfg.getUpdateUserId());
if (T.ObjectUtil.isNotEmpty(updateUser)) {
cfg.setCreateUser(updateUser);
}
WorkspaceEntity workspace = workspaceService.getById(cfg.getWorkspaceId());
if (T.ObjectUtil.isNotEmpty(workspace)) {
cfg.setWorkspace(workspace);
}
JobEntity lastJob = jobService.getOne(new LambdaQueryWrapper<JobEntity>()
.eq(JobEntity::getJobCfgId, cfg.getId())
.orderByDesc(JobEntity::getCreateTimestamp)
.last("limit 1"));
if (T.ObjectUtil.isNotEmpty(lastJob)) {
cfg.setLastJobResult(lastJob);
List<PcapEntity> pcapList = pcapService.list(new LambdaQueryWrapper<PcapEntity>().eq(PcapEntity::getJobId, lastJob.getId()));
cfg.setPcap(pcapList);
}
return cfg;
}
@Override
public Page queryList(Map<String, Object> params) {
Page page = new Query(JobCfgEntity.class).getPage(params);
List<JobCfgEntity> jobList = this.getBaseMapper().queryList(page, params);
page.setRecords(jobList);
return page;
}
private void validateJobCfgInfo(JobCfgEntity cfg) {
if (!Constants.JOB_CFG_STATUS_LIST.contains(cfg.getStatus())) {
throw new ASWException(RCode.JOB_CFG_STATUS_ERROR);
}
if (!Constants.JOB_CFG_TYPE_LIST.contains(cfg.getType())) {
throw new ASWException(RCode.JOB_CFG_TYPE_ERROR);
}
if (T.StrUtil.equals(cfg.getType(), "cron") && cfg.getCron() == null) {
throw new ASWException(RCode.JOB_CFG_CRON_CANNOT_EMPTY);
}
if (T.StrUtil.isNotEmpty(cfg.getCron()) && !CronExpression.isValidExpression(cfg.getCron())) {
throw new ASWException(RCode.JOB_CFG_CRON_ERROR);
}
JobCfgEntity entity = this.getOne(new LambdaQueryWrapper<JobCfgEntity>().eq(JobCfgEntity::getName, cfg.getName()));
if (cfg.getId() == null && T.ObjectUtil.isNotEmpty(entity)) {
throw new ASWException(RCode.JOB_CFG_NANE_ALREADY_EXISTS);
}
if (cfg.getId() != null && T.ObjectUtil.isNotEmpty(entity) && !T.StrUtil.equals(cfg.getId(), entity.getId())) {
throw new ASWException(RCode.JOB_CFG_NANE_ALREADY_EXISTS);
}
}
@Override
public JobCfgEntity saveJobCfg(String workspaceId, JobCfgEntity cfg) {
this.validateJobCfgInfo(cfg);
cfg.setCreateTimestamp(System.currentTimeMillis());
cfg.setUpdateTimestamp(System.currentTimeMillis());
cfg.setCreateUserId(StpUtil.getLoginIdAsString());
cfg.setUpdateUserId(StpUtil.getLoginIdAsString());
cfg.setWorkspaceId(workspaceId);
this.save(cfg);
return cfg;
}
@Override
public JobCfgEntity updateCfg(String workspaceId, JobCfgEntity cfg) {
this.validateJobCfgInfo(cfg);
cfg.setUpdateTimestamp(System.currentTimeMillis());
cfg.setUpdateUserId(StpUtil.getLoginIdAsString());
this.updateById(cfg);
return cfg;
}
}

View File

@@ -0,0 +1,203 @@
package net.geedge.asw.module.job.service.impl;
import cn.dev33.satoken.stp.StpUtil;
import cn.hutool.log.Log;
import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
import com.baomidou.mybatisplus.core.conditions.update.LambdaUpdateWrapper;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;
import net.geedge.asw.common.config.Query;
import net.geedge.asw.common.util.*;
import net.geedge.asw.module.app.entity.PackageEntity;
import net.geedge.asw.module.app.service.IPackageService;
import net.geedge.asw.module.environment.entity.EnvironmentEntity;
import net.geedge.asw.module.environment.entity.EnvironmentSessionEntity;
import net.geedge.asw.module.environment.service.IEnvironmentService;
import net.geedge.asw.module.environment.service.IEnvironmentSessionService;
import net.geedge.asw.module.job.dao.JobDao;
import net.geedge.asw.module.job.entity.JobCfgEntity;
import net.geedge.asw.module.job.entity.JobEntity;
import net.geedge.asw.module.job.entity.PcapEntity;
import net.geedge.asw.module.job.entity.PlaybookEntity;
import net.geedge.asw.module.job.service.IJobCfgService;
import net.geedge.asw.module.job.service.IJobService;
import net.geedge.asw.module.job.service.IPcapService;
import net.geedge.asw.module.job.service.IPlaybookService;
import net.geedge.asw.module.job.util.JobConstant;
import net.geedge.asw.module.sys.entity.SysUserEntity;
import net.geedge.asw.module.sys.service.ISysUserService;
import net.geedge.asw.module.workspace.entity.WorkspaceEntity;
import net.geedge.asw.module.workspace.service.IWorkspaceService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
import java.io.File;
import java.io.IOException;
import java.io.RandomAccessFile;
import java.util.List;
import java.util.Map;
@Service
public class JobServiceImpl extends ServiceImpl<JobDao, JobEntity> implements IJobService {
private static final Log log = Log.get();
@Autowired
private IEnvironmentService environmentService;
@Autowired
private IPlaybookService playbookService;
@Autowired
private IPackageService packageService;
@Autowired
private IPcapService pcapService;
@Autowired
private ISysUserService userService;
@Autowired
private IWorkspaceService workspaceService;
@Autowired
private IEnvironmentSessionService environmentSessionService;
@Autowired
private IJobCfgService jobCfgService;
@Value("${asw.resources.path:resources}")
private String resources;
@Override
public JobEntity queryInfo(String id) {
JobEntity job = this.getById(id);
T.VerifyUtil.is(job).notNull(RCode.SYS_RECORD_NOT_FOUND);
EnvironmentEntity env = environmentService.getById(job.getEnvId());
job.setEnvironment(env);
PlaybookEntity playbook = playbookService.getById(job.getPlaybookId());
job.setPlaybook(playbook);
PackageEntity pkg = packageService.getById(job.getPackageId());
job.setPkg(pkg);
List<PcapEntity> pcapList = pcapService.list(new LambdaQueryWrapper<PcapEntity>().eq(PcapEntity::getJobId, id));
job.setPcap(pcapList);
SysUserEntity createUser = userService.getById(job.getCreateUserId());
job.setCreateUser(createUser);
WorkspaceEntity workspace = workspaceService.getById(job.getWorkspaceId());
job.setWorkspace(workspace);
EnvironmentSessionEntity session = environmentSessionService.getOne(new LambdaQueryWrapper<EnvironmentSessionEntity>().eq(EnvironmentSessionEntity::getJobId, id));
job.setSession(session);
JobCfgEntity cfg = jobCfgService.getById(job.getJobCfgId());
job.setJobCfg(cfg);
if (-1 == job.getStartTimestamp()) {
job.setStartTimestamp(null);
}
if (-1 == job.getEndTimestamp()) {
job.setEndTimestamp(null);
}
return job;
}
@Override
public Page queryList(Map<String, Object> params) {
Page page = new Query(JobEntity.class).getPage(params);
List<JobEntity> jobList = this.getBaseMapper().queryList(page, params);
jobList.stream().forEach(jobEntity -> {
if (-1 == jobEntity.getStartTimestamp()) {
jobEntity.setStartTimestamp(null);
}
if (-1 == jobEntity.getEndTimestamp()) {
jobEntity.setEndTimestamp(null);
}
});
page.setRecords(jobList);
return page;
}
@Override
@Transactional(rollbackFor = Exception.class)
public JobEntity saveJob(String workspaceId, String jobCfgId) {
JobCfgEntity cfg = jobCfgService.getById(jobCfgId);
if (T.ObjectUtil.isEmpty(cfg)){
throw new ASWException(RCode.JOB_CFG_NOT_EXIST);
}
JobEntity job = new JobEntity();
job.setWorkspaceId(workspaceId);
job.setJobCfgId(jobCfgId);
job.setParameters(cfg.getParameters());
job.setEnvId(cfg.getEnvId());
job.setPackageId(cfg.getPackageId());
job.setPlaybookId(cfg.getPlaybookId());
job.setCreateTimestamp(System.currentTimeMillis());
job.setUpdateTimestamp(System.currentTimeMillis());
job.setCreateUserId(StpUtil.getLoginIdAsString());
job.setUpdateUserId(StpUtil.getLoginIdAsString());
job.setStatus(JobConstant.JobStatus.CREATED.getValue());
// save
this.save(job);
// trace log file path
String saveFileName = job.getId() + "." + "log";
File traceLogFile = FileResourceUtil.createFile(resources, job.getWorkspaceId(), Constants.FileTypeEnum.JOB.getType(), job.getId(), saveFileName);
this.update(new LambdaUpdateWrapper<JobEntity>()
.set(JobEntity::getLogPath, traceLogFile.getPath())
.eq(JobEntity::getId, job.getId()));
return job;
}
@Override
@Transactional(rollbackFor = Exception.class)
public void removeJob(List<String> ids) {
// remove
this.removeBatchByIds(ids);
}
@Override
public Map queryJobLog(String id, Integer offset) {
JobEntity job = this.getById(id);
Map result = T.MapUtil.newHashMap();
File logFile = T.FileUtil.file(job.getLogPath());
if (logFile.exists()){
try (RandomAccessFile raf = new RandomAccessFile(logFile, "r")) {
if (offset > raf.length()){
result.put("content", T.StrUtil.EMPTY);
result.put("length", 0);
result.put("offset", offset);
}else {
raf.seek(offset);
byte[] bytes = new byte[(int)raf.length() - offset];
raf.readFully(bytes);
String content = new String(bytes);
result.put("content", content);
result.put("length", bytes.length);
result.put("offset", offset + bytes.length);
}
} catch (IOException e) {
log.error("queryJobLog error", e);
throw new ASWException(RCode.ERROR);
}
}else {
result.put("content", T.StrUtil.EMPTY);
result.put("length", 0);
result.put("offset", 0);
}
return result;
}
}

View File

@@ -1,37 +1,37 @@
package net.geedge.asw.module.runner.service.impl; package net.geedge.asw.module.job.service.impl;
import cn.dev33.satoken.stp.SaTokenInfo; import cn.dev33.satoken.stp.SaTokenInfo;
import cn.dev33.satoken.stp.StpUtil; import cn.dev33.satoken.stp.StpUtil;
import cn.hutool.core.net.URLEncodeUtil; import cn.hutool.core.net.URLEncodeUtil;
import cn.hutool.core.net.url.UrlBuilder; import cn.hutool.core.net.url.UrlBuilder;
import cn.hutool.extra.template.Template;
import cn.hutool.log.Log; import cn.hutool.log.Log;
import com.alibaba.fastjson2.JSONArray; import com.alibaba.fastjson2.JSONArray;
import com.alibaba.fastjson2.JSONObject; import com.alibaba.fastjson2.JSONObject;
import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper; import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
import com.baomidou.mybatisplus.core.conditions.update.LambdaUpdateWrapper;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page; import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl; import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;
import net.geedge.asw.common.config.Query;
import net.geedge.asw.common.config.SpringContextUtils; import net.geedge.asw.common.config.SpringContextUtils;
import net.geedge.asw.common.util.ASWException; import net.geedge.asw.common.util.*;
import net.geedge.asw.common.util.RCode;
import net.geedge.asw.common.util.T;
import net.geedge.asw.module.app.entity.PackageEntity; import net.geedge.asw.module.app.entity.PackageEntity;
import net.geedge.asw.module.app.service.IApplicationService;
import net.geedge.asw.module.app.service.IPackageService; import net.geedge.asw.module.app.service.IPackageService;
import net.geedge.asw.module.environment.entity.EnvironmentEntity; import net.geedge.asw.module.environment.entity.EnvironmentEntity;
import net.geedge.asw.module.environment.service.IEnvironmentService; import net.geedge.asw.module.environment.service.IEnvironmentService;
import net.geedge.asw.module.feign.client.DashboardClient;
import net.geedge.asw.module.feign.client.KibanaClient; import net.geedge.asw.module.feign.client.KibanaClient;
import net.geedge.asw.module.runner.dao.PcapDao; import net.geedge.asw.module.job.dao.PcapDao;
import net.geedge.asw.module.runner.entity.JobEntity; import net.geedge.asw.module.job.entity.JobEntity;
import net.geedge.asw.module.runner.entity.PcapEntity; import net.geedge.asw.module.job.entity.PcapEntity;
import net.geedge.asw.module.runner.entity.PlaybookEntity; import net.geedge.asw.module.job.entity.PlaybookEntity;
import net.geedge.asw.module.runner.service.IJobService; import net.geedge.asw.module.job.service.IJobService;
import net.geedge.asw.module.runner.service.IPcapService; import net.geedge.asw.module.job.service.IPcapService;
import net.geedge.asw.module.runner.service.IPlaybookService; import net.geedge.asw.module.job.service.IPlaybookService;
import net.geedge.asw.module.runner.util.PcapParserThread; import net.geedge.asw.module.job.util.PcapParserThread;
import net.geedge.asw.module.runner.util.RunnerConstant; import net.geedge.asw.module.job.util.JobConstant;
import net.geedge.asw.module.workbook.service.IWorkbookResourceService; import net.geedge.asw.module.sys.entity.SysUserEntity;
import net.geedge.asw.module.workbook.util.WorkbookConstant; import net.geedge.asw.module.sys.service.ISysConfigService;
import net.geedge.asw.module.sys.service.ISysUserService;
import net.geedge.asw.module.workspace.entity.WorkspaceEntity; import net.geedge.asw.module.workspace.entity.WorkspaceEntity;
import net.geedge.asw.module.workspace.service.IWorkspaceService; import net.geedge.asw.module.workspace.service.IWorkspaceService;
import org.apache.commons.io.FileUtils; import org.apache.commons.io.FileUtils;
@@ -74,30 +74,39 @@ public class PcapServiceImpl extends ServiceImpl<PcapDao, PcapEntity> implements
@Autowired @Autowired
private IPackageService packageService; private IPackageService packageService;
@Autowired
private IApplicationService applicationService;
@Autowired
private IWorkbookResourceService workbookResourceService;
@Autowired @Autowired
private IWorkspaceService workspaceService; private IWorkspaceService workspaceService;
@Autowired
private ISysConfigService configService;
@Autowired
private ISysUserService userService;
@Value("${kibana.url:127.0.0.1:5601}") @Value("${kibana.url:127.0.0.1:5601}")
private String kibanaUrl; private String kibanaUrl;
@jakarta.annotation.Resource @jakarta.annotation.Resource
private KibanaClient kibanaClient; private KibanaClient kibanaClient;
@jakarta.annotation.Resource
private DashboardClient dashboardClient;
@Value("${controller.url:http://127.0.0.1}") @Value("${controller.url:http://127.0.0.1}")
private String aswControllerUrl; private String aswControllerUrl;
@Value("${asw.resources.path:resources}")
private String resources;
@Override @Override
public PcapEntity queryInfo(String id) { public PcapEntity queryInfo(String id) {
PcapEntity pcap = this.getById(id); PcapEntity pcap = this.getById(id);
T.VerifyUtil.is(pcap).notNull(RCode.SYS_RECORD_NOT_FOUND); T.VerifyUtil.is(pcap).notNull(RCode.SYS_RECORD_NOT_FOUND);
JobEntity job = jobService.getOne(new LambdaQueryWrapper<JobEntity>().eq(JobEntity::getPcapId, pcap.getId())); SysUserEntity user = userService.getById(pcap.getCreateUserId());
pcap.setCreateUser(user);
JobEntity job = jobService.getOne(new LambdaQueryWrapper<JobEntity>().eq(JobEntity::getId, pcap.getJobId()));
if (T.ObjectUtil.isNotNull(job)) { if (T.ObjectUtil.isNotNull(job)) {
pcap.setJobId(job.getId()); pcap.setJobId(job.getId());
@@ -115,7 +124,7 @@ public class PcapServiceImpl extends ServiceImpl<PcapDao, PcapEntity> implements
@Override @Override
public Page queryList(Map<String, Object> params) { public Page queryList(Map<String, Object> params) {
Page page = T.PageUtil.getPage(params); Page page = new Query(PcapEntity.class).getPage(params);
List<PcapEntity> pcapList = this.getBaseMapper().queryList(page, params); List<PcapEntity> pcapList = this.getBaseMapper().queryList(page, params);
page.setRecords(pcapList); page.setRecords(pcapList);
return page; return page;
@@ -139,7 +148,7 @@ public class PcapServiceImpl extends ServiceImpl<PcapDao, PcapEntity> implements
byte[] bytes = fileResource.getInputStream().readAllBytes(); byte[] bytes = fileResource.getInputStream().readAllBytes();
entity.setSize((long) bytes.length); entity.setSize((long) bytes.length);
entity.setStatus(RunnerConstant.PcapStatus.UPLOADED.getValue()); entity.setStatus(JobConstant.PcapStatus.UPLOADED.getValue());
entity.setCreateTimestamp(System.currentTimeMillis()); entity.setCreateTimestamp(System.currentTimeMillis());
entity.setCreateUserId(createUserId); entity.setCreateUserId(createUserId);
entity.setWorkspaceId(workspaceId); entity.setWorkspaceId(workspaceId);
@@ -147,7 +156,7 @@ public class PcapServiceImpl extends ServiceImpl<PcapDao, PcapEntity> implements
// path // path
String fileExtName = T.StrUtil.emptyToDefault(T.FileUtil.extName(fileResource.getFilename()), "pcap"); String fileExtName = T.StrUtil.emptyToDefault(T.FileUtil.extName(fileResource.getFilename()), "pcap");
String saveFileName = pcapId + "." + fileExtName; String saveFileName = pcapId + "." + fileExtName;
File destination = T.FileUtil.file(T.WebPathUtil.getRootPath(), workspaceId, saveFileName); File destination = FileResourceUtil.createFile(resources, workspaceId, Constants.FileTypeEnum.PCAP.getType(), pcapId, saveFileName);
FileUtils.copyInputStreamToFile(fileResource.getInputStream(), destination); FileUtils.copyInputStreamToFile(fileResource.getInputStream(), destination);
entity.setPath(destination.getPath()); entity.setPath(destination.getPath());
@@ -156,10 +165,10 @@ public class PcapServiceImpl extends ServiceImpl<PcapDao, PcapEntity> implements
entity.setMd5(md5Hex); entity.setMd5(md5Hex);
// 根据文件 md5值 判断是否已上存在存在则响应当前实体 // 根据文件 md5值 判断是否已上存在存在则响应当前实体
PcapEntity findPcapByMd5 = this.getOne(new LambdaQueryWrapper<PcapEntity>().eq(PcapEntity::getMd5, md5Hex).eq(PcapEntity::getWorkspaceId, workspaceId)); /*PcapEntity findPcapByMd5 = this.getOne(new LambdaQueryWrapper<PcapEntity>().eq(PcapEntity::getMd5, md5Hex).eq(PcapEntity::getWorkspaceId, workspaceId));
if (T.ObjectUtil.isNotNull(findPcapByMd5)) { if (T.ObjectUtil.isNotNull(findPcapByMd5)) {
return findPcapByMd5; return findPcapByMd5;
} }*/
// save // save
this.save(entity); this.save(entity);
@@ -177,16 +186,11 @@ public class PcapServiceImpl extends ServiceImpl<PcapDao, PcapEntity> implements
PcapEntity pcap = this.getById(id); PcapEntity pcap = this.getById(id);
// remove file // remove file
T.FileUtil.del(pcap.getPath()); T.FileUtil.del(pcap.getPath());
T.FileUtil.del(pcap.getCommonPcapFilePath()); Map summary = T.JSONUtil.toBean(pcap.getSummary(), Map.class);
T.FileUtil.del(T.MapUtil.getStr(summary, "commentPath"));
// remove // remove
this.removeById(id); this.removeById(id);
// update job pcap_id
jobService.update(new LambdaUpdateWrapper<JobEntity>()
.set(JobEntity::getPcapId, "")
.eq(JobEntity::getPcapId, id)
);
} }
} }
@@ -199,6 +203,7 @@ public class PcapServiceImpl extends ServiceImpl<PcapDao, PcapEntity> implements
Properties properties = new Properties(); Properties properties = new Properties();
properties.setProperty("sharkdApiHostAddr", this.sharkdApiHostAddr); properties.setProperty("sharkdApiHostAddr", this.sharkdApiHostAddr);
properties.setProperty("aswControllerUrl", this.aswControllerUrl); properties.setProperty("aswControllerUrl", this.aswControllerUrl);
properties.setProperty("resources", this.resources);
for (String id : ids) { for (String id : ids) {
PcapEntity pcapEntity = this.getById(id); PcapEntity pcapEntity = this.getById(id);
@@ -265,11 +270,12 @@ public class PcapServiceImpl extends ServiceImpl<PcapDao, PcapEntity> implements
log.error("delete openSearch index error index: {}", indexName); log.error("delete openSearch index error index: {}", indexName);
throw new RuntimeException("delete openSearch index error ", e); throw new RuntimeException("delete openSearch index error ", e);
} }
pcapEntity.setStatus(RunnerConstant.PcapStatus.UPLOADED.getValue()); pcapEntity.setStatus(JobConstant.PcapStatus.UPLOADED.getValue());
// del common pcap file
Map summary = T.JSONUtil.toBean(pcapEntity.getSummary(), Map.class);
T.FileUtil.del(T.MapUtil.getStr(summary, "commentPath"));
// reset summary // reset summary
pcapEntity.setSummary("{}"); pcapEntity.setSummary("{}");
// del common pcap file
T.FileUtil.del(pcapEntity.getCommonPcapFilePath().toFile());
this.updateById(pcapEntity); this.updateById(pcapEntity);
} }
} }
@@ -334,6 +340,14 @@ public class PcapServiceImpl extends ServiceImpl<PcapDao, PcapEntity> implements
kibanaClient.saveIndexPattern(token, workspaceId, body); kibanaClient.saveIndexPattern(token, workspaceId, body);
} }
// get index-patten id
String indexId = savedObjects.stream()
.map(obj -> {
return ((JSONObject) obj).getString("id");
})
.findFirst()
.get();
// build url // build url
String baseUrl = UrlBuilder.ofHttp(kibanaUrl) String baseUrl = UrlBuilder.ofHttp(kibanaUrl)
.addPath("/app/data-explorer/discover") .addPath("/app/data-explorer/discover")
@@ -341,7 +355,7 @@ public class PcapServiceImpl extends ServiceImpl<PcapDao, PcapEntity> implements
.toString(); .toString();
// build query param // build query param
String param1 = String.format("_a=(discover:(columns:!(_source),isDirty:!f,sort:!()),metadata:(indexPattern:'%s',view:discover))", workspaceId); String param1 = String.format("_a=(discover:(columns:!(_source),isDirty:!f,sort:!()),metadata:(indexPattern:'%s',view:discover))", indexId);
String param2 = "_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now-15m,to:now))"; String param2 = "_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now-15m,to:now))";
String filter = pcapList.stream() String filter = pcapList.stream()
@@ -384,8 +398,90 @@ public class PcapServiceImpl extends ServiceImpl<PcapDao, PcapEntity> implements
SaTokenInfo tokenInfo = StpUtil.getTokenInfo(); SaTokenInfo tokenInfo = StpUtil.getTokenInfo();
String token = tokenInfo.getTokenValue(); String token = tokenInfo.getTokenValue();
String dashboardName = String.format("dashboard-%s", workspace.getName());
String indexName = String.format("workspace-%s-*", workspace.getName());
// check if dashboard exists
JSONObject dashboard = kibanaClient.findIndexPattern(token, "dashboard" ,dashboardName);
JSONArray dashboardObjects = dashboard.getJSONArray("saved_objects");
boolean dashboardExists = dashboardObjects.stream()
.filter(obj -> {
JSONObject attributes = ((JSONObject) obj).getJSONObject("attributes");
if (T.ObjectUtil.isEmpty(attributes)) return false;
String title = attributes.getString("title");
return T.StrUtil.equals(dashboardName, title);
})
.findFirst()
.isPresent();
File dashboardFile = null;
try {
if (!dashboardExists) {
if (log.isDebugEnabled()) {
log.debug("[generateKibanaDashboardUrl] [dashboard: {}] [exists: {}]", dashboardName, dashboardExists);
}
// check if index-pattern exists
JSONObject indexPattern = kibanaClient.findIndexPattern(token, "index-pattern", indexName);
JSONArray savedObjects = indexPattern.getJSONArray("saved_objects");
boolean indexExists = savedObjects.stream()
.filter(obj -> {
JSONObject attributes = ((JSONObject) obj).getJSONObject("attributes");
if (T.ObjectUtil.isEmpty(attributes)) return false;
String title = attributes.getString("title");
return T.StrUtil.equals(indexName, title);
})
.findFirst()
.isPresent();
if (indexExists){
// delete index-pattern
String indexId = savedObjects.stream().map(x -> {
return ((JSONObject) x).getString("id");
}).findFirst().get();
kibanaClient.deleteIndexPattern(token, indexId, true);
}
// import dashboard
Map<Object, Object> params = T.MapUtil.builder()
.put("indexName", indexName)
.put("dashboardName", dashboardName)
.build();
// render dashboard template
String opensearchDashboardTemplate = configService.getValue("opensearch_dashboard_template");
Template template = T.TemplateUtil.createEngine().getTemplate(opensearchDashboardTemplate);
opensearchDashboardTemplate = template.render(params);
dashboardFile = T.FileUtil.file(Constants.TEMP_PATH, "dashboard_template.ndjson");
T.FileUtil.writeString(opensearchDashboardTemplate, dashboardFile, "utf-8");
// create dashboard
dashboardClient.importDashboard(token, dashboardFile, true);
// get dashboardId
JSONObject dashboardObj = kibanaClient.findIndexPattern(token, "dashboard", dashboardName);
JSONArray dashboards = dashboardObj.getJSONArray("saved_objects");
String dashboardId = dashboards.stream()
.filter(x -> {
JSONObject attributes = ((JSONObject) x).getJSONObject("attributes");
if (T.ObjectUtil.isEmpty(attributes)) return false;
String title = attributes.getString("title");
return T.StrUtil.equals(dashboardName, title);
})
.map(x -> {
return ((JSONObject) x).getString("id");
}).findFirst().get();
workspace.setProperties(T.JSONUtil.toJsonStr(Map.of("dashboardId", dashboardId)));
workspaceService.updateById(workspace);
}
} finally {
T.FileUtil.del(dashboardFile);
}
String dashboardId = T.JSONUtil.parseObj(workspace.getProperties()).getStr("dashboardId"); String dashboardId = T.JSONUtil.parseObj(workspace.getProperties()).getStr("dashboardId");
String dashboardName = String.format("workspace-%s", workspace.getName());
// build url // build url
String baseUrl = UrlBuilder.ofHttp(kibanaUrl) String baseUrl = UrlBuilder.ofHttp(kibanaUrl)

View File

@@ -0,0 +1,139 @@
package net.geedge.asw.module.job.service.impl;
import cn.dev33.satoken.stp.StpUtil;
import cn.hutool.log.Log;
import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;
import net.geedge.asw.common.config.Query;
import net.geedge.asw.common.util.*;
import net.geedge.asw.module.job.dao.PlaybookDao;
import net.geedge.asw.module.job.entity.PlaybookEntity;
import net.geedge.asw.module.job.service.IPlaybookService;
import org.apache.commons.io.FileUtils;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
import org.springframework.web.multipart.MultipartFile;
import java.io.File;
import java.util.Arrays;
import java.util.List;
import java.util.Map;
@Service
public class PlaybookServiceImpl extends ServiceImpl<PlaybookDao, PlaybookEntity> implements IPlaybookService {
private final static Log log = Log.get();
@Value("${asw.resources.path:resources}")
private String resources;
@Override
public PlaybookEntity detail(String workspaceId, String id) {
PlaybookEntity playbook = this.baseMapper.queryInfo(workspaceId, id);
return playbook;
}
@Override
public Page queryList(String workspaceId, Map params) {
Page page = new Query(PlaybookEntity.class).getPage(params);
List<PlaybookEntity> playbookList = this.baseMapper.queryList(page, workspaceId, params);
page.setRecords(playbookList);
return page;
}
@Override
@Transactional(rollbackFor = Exception.class)
public PlaybookEntity savePlaybook(String workspaceId, MultipartFile file, String name, String type, String description) {
List<PlaybookEntity> playbookList = this.baseMapper.selectList(new LambdaQueryWrapper<PlaybookEntity>().eq(PlaybookEntity::getWorkspaceId, workspaceId).eq(PlaybookEntity::getName, name));
if (T.CollUtil.isNotEmpty(playbookList)) {
throw new ASWException(RCode.PLAYBOOK_NAME_DUPLICATE);
}
PlaybookEntity playbook = new PlaybookEntity();
String playbookId = T.StrUtil.uuid();
String fileExtName = T.StrUtil.emptyToDefault(T.FileUtil.extName(file.getName()), "zip");
String saveFileName = playbookId + "." + fileExtName;
File destination = FileResourceUtil.createFile(resources, workspaceId, Constants.FileTypeEnum.PLAYBOOK.getType(), playbookId, saveFileName);
File unzip = null;
try {
playbook.setId(playbookId);
playbook.setWorkspaceId(workspaceId);
playbook.setName(name);
playbook.setDescription(description);
playbook.setCreateUserId(StpUtil.getLoginIdAsString());
playbook.setUpdateUserId(StpUtil.getLoginIdAsString());
playbook.setCreateTimestamp(System.currentTimeMillis());
playbook.setUpdateTimestamp(System.currentTimeMillis());
// path
FileUtils.copyInputStreamToFile(file.getInputStream(), destination);
playbook.setPath(destination.getPath());
if (T.StrUtil.equalsIgnoreCase(type, "airtest_script")){
playbook.setType("airtest_script");
unzip = T.ZipUtil.unzip(destination);
List<File> fileList = Arrays.stream(unzip.listFiles()).toList();
if (T.CollUtil.isEmpty(fileList) || fileList.size() != 1) {
throw new ASWException(RCode.PLAYBOOK_INVALID_FILE);
}
File playbookDir = fileList.getFirst();
String dirName = playbookDir.getName();
if (!dirName.endsWith(".air")) {
throw new ASWException(RCode.PLAYBOOK_INVALID_FILE);
}
String pyName = dirName.replace("air", "py");
List<String> fileNameList = Arrays.stream(playbookDir.list()).toList();
if (T.CollUtil.isEmpty(fileNameList) || !fileNameList.contains(pyName)) {
throw new ASWException(RCode.PLAYBOOK_INVALID_FILE);
}
}
if (T.StrUtil.equalsIgnoreCase(type, "python")){
playbook.setType("python");
unzip = T.ZipUtil.unzip(destination);
List<File> fileList = Arrays.stream(unzip.listFiles()).toList();
fileList = fileList.stream().filter(x -> {
return T.StrUtil.equals("main.py", x.getName());
}).toList();
if (T.CollUtil.isEmpty(fileList)) {
throw new ASWException(RCode.PLAYBOOK_INVALID_FILE);
}
}
this.save(playbook);
} catch (Exception e) {
log.error(e, "[savePlaybook] [error] [file: {}]", file.getName());
T.FileUtil.del(destination);
throw new ASWException(RCode.PLAYBOOK_INVALID_FILE);
}finally {
T.FileUtil.del(unzip);
}
return playbook;
}
@Override
public void delete(String workspaceId, String ids) {
List<String> idList = Arrays.asList(ids.split(","));
for (String id : idList) {
PlaybookEntity entity = this.getById(id);
// remove file
T.FileUtil.del(entity.getPath());
// remove
this.removeById(id);
}
}
@Override
public PlaybookEntity updatePlaybook(String workspaceId, String playbookId, String name, String description) {
PlaybookEntity playbook = this.baseMapper.selectOne(new LambdaQueryWrapper<PlaybookEntity>().eq(PlaybookEntity::getWorkspaceId, workspaceId).eq(PlaybookEntity::getName, name));
if (T.ObjectUtil.isNotEmpty(playbook) && !playbookId.equals(playbook.getId())) {
throw new ASWException(RCode.PLAYBOOK_NAME_DUPLICATE);
}
PlaybookEntity playbookEntity = this.getById(playbookId);
playbookEntity.setName(name);
playbookEntity.setDescription(description);
this.updateById(playbookEntity);
return playbookEntity;
}
}

View File

@@ -1,6 +1,6 @@
package net.geedge.asw.module.runner.util; package net.geedge.asw.module.job.util;
public class RunnerConstant { public class JobConstant {
/** /**

View File

@@ -0,0 +1,61 @@
package net.geedge.asw.module.job.util;
import net.geedge.asw.common.util.T;
import net.geedge.asw.module.job.entity.JobEntity;
import org.springframework.stereotype.Component;
import java.util.*;
import java.util.concurrent.ConcurrentHashMap;
/**
* job 任务队列管理
*/
@Component
public class JobQueueManager {
// 每个环境对应一个任务队列,以创建时间排序
private final Map<String, PriorityQueue<JobEntity>> groupedQueues = new ConcurrentHashMap<>();
// 添加任务到队列
public synchronized void addJob(JobEntity job) {
boolean contains = groupedQueues.containsKey(job.getEnvId());
if (!contains) {
PriorityQueue<JobEntity> queue = new PriorityQueue<>(Comparator.comparing(JobEntity::getCreateTimestamp));
queue.offer(job);
groupedQueues.put(job.getEnvId(), queue);
} else {
groupedQueues.get(job.getEnvId()).offer(job);
}
}
// 获取每个环境下的任务
public synchronized List<JobEntity> fetchNextJob() {
List<JobEntity> list = T.ListUtil.list(false);
for (Map.Entry<String, PriorityQueue<JobEntity>> queueEntry : groupedQueues.entrySet()) {
PriorityQueue<JobEntity> queue = queueEntry.getValue();
if (queue == null || queue.isEmpty()) {
continue; // 如果该环境下没有任务则跳过
}
list.add(queue.poll());
}
return list;
}
// 检查所有队列是否为空
public synchronized boolean isAllQueuesEmpty() {
return groupedQueues.values().stream().allMatch(PriorityQueue::isEmpty);
}
// 重新添加队列
public synchronized void requeueJob(JobEntity job) {
PriorityQueue<JobEntity> queue = groupedQueues.get(job.getEnvId());
queue.offer(job);
}
// 删除
public synchronized void removeJob(JobEntity job) {
PriorityQueue<JobEntity> queue = groupedQueues.get(job.getEnvId());
queue.remove(job);
}
}

View File

@@ -1,4 +1,4 @@
package net.geedge.asw.module.runner.util; package net.geedge.asw.module.job.util;
import cn.hutool.log.Log; import cn.hutool.log.Log;
import com.alibaba.fastjson2.JSONArray; import com.alibaba.fastjson2.JSONArray;
@@ -12,8 +12,8 @@ import net.geedge.asw.common.util.T;
import net.geedge.asw.module.feign.client.GeoipClient; import net.geedge.asw.module.feign.client.GeoipClient;
import net.geedge.asw.module.feign.client.PcapCommentClient; import net.geedge.asw.module.feign.client.PcapCommentClient;
import net.geedge.asw.module.feign.client.ZeekClient; import net.geedge.asw.module.feign.client.ZeekClient;
import net.geedge.asw.module.runner.entity.PcapEntity; import net.geedge.asw.module.job.entity.PcapEntity;
import net.geedge.asw.module.runner.service.IPcapService; import net.geedge.asw.module.job.service.IPcapService;
import org.apache.commons.io.FileUtils; import org.apache.commons.io.FileUtils;
import org.apache.commons.lang3.time.StopWatch; import org.apache.commons.lang3.time.StopWatch;
import org.opensearch.client.opensearch.OpenSearchClient; import org.opensearch.client.opensearch.OpenSearchClient;
@@ -35,7 +35,7 @@ import java.time.format.DateTimeFormatter;
import java.util.*; import java.util.*;
import java.util.stream.Collectors; import java.util.stream.Collectors;
import static net.geedge.asw.module.runner.util.RunnerConstant.PcapStatus; import static net.geedge.asw.module.job.util.JobConstant.PcapStatus;
@Data @Data
public class PcapParserThread implements Runnable { public class PcapParserThread implements Runnable {
@@ -106,7 +106,7 @@ public class PcapParserThread implements Runnable {
} }
// file path: /{path}/{workspace_id}/pcap_comment/{pcap_id}.pcapng // file path: /{path}/{workspace_id}/pcap_comment/{pcap_id}.pcapng
File destination = pcapEntity.getCommonPcapFilePath().toFile(); File destination = pcapEntity.getCommonPcapFilePath(properties.getProperty("resources"));
T.FileUtil.del(destination); T.FileUtil.del(destination);
FileUtils.copyInputStreamToFile(response.body().asInputStream(), destination); FileUtils.copyInputStreamToFile(response.body().asInputStream(), destination);
} }
@@ -219,7 +219,7 @@ public class PcapParserThread implements Runnable {
.put("sessions", jsonArray.size()) .put("sessions", jsonArray.size())
.put("packets", packets) .put("packets", packets)
.put("services", services) .put("services", services)
.put("commentPath", pcapEntity.getCommonPcapFilePath().toString()) .put("commentPath", pcapEntity.getCommonPcapFilePath(properties.getProperty("resources")).getAbsolutePath())
.build(); .build();
pcapService.update(new LambdaUpdateWrapper<PcapEntity>() pcapService.update(new LambdaUpdateWrapper<PcapEntity>()
.set(PcapEntity::getSummary, T.JSONUtil.toJsonStr(m)) .set(PcapEntity::getSummary, T.JSONUtil.toJsonStr(m))

View File

@@ -1,4 +1,4 @@
package net.geedge.asw.module.runner.util; package net.geedge.asw.module.job.util;
import cn.hutool.json.JSONArray; import cn.hutool.json.JSONArray;
import cn.hutool.json.JSONConfig; import cn.hutool.json.JSONConfig;

View File

@@ -1,4 +1,4 @@
package net.geedge.asw.module.runner.util; package net.geedge.asw.module.job.util;
import cn.hutool.log.Log; import cn.hutool.log.Log;
import lombok.Data; import lombok.Data;

View File

@@ -1,80 +0,0 @@
package net.geedge.asw.module.runner.controller;
import com.baomidou.mybatisplus.core.conditions.update.LambdaUpdateWrapper;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import net.geedge.asw.common.util.R;
import net.geedge.asw.common.util.RCode;
import net.geedge.asw.common.util.T;
import net.geedge.asw.module.runner.entity.JobEntity;
import net.geedge.asw.module.runner.service.IJobService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.*;
import java.util.Arrays;
import java.util.List;
import java.util.Map;
@RestController
@RequestMapping("/api/v1/workspace")
public class JobController {
@Autowired
private IJobService jobService;
@GetMapping("/{workspaceId}/job/{id}")
public R detail(@PathVariable("workspaceId") String workspaceId,
@PathVariable("id") String id) {
JobEntity jobEntity = jobService.queryInfo(id);
return R.ok().putData("record", jobEntity);
}
@GetMapping("/{workspaceId}/job")
public R list(@PathVariable("workspaceId") String workspaceId,
@RequestParam Map<String, Object> params) {
T.VerifyUtil.is(params).notNull()
.and(workspaceId).notEmpty(RCode.WORKSPACE_ID_CANNOT_EMPTY);
params.put("workspaceId", workspaceId);
Page page = jobService.queryList(params);
return R.ok(page);
}
@PostMapping("/{workspaceId}/job")
public R add(@PathVariable("workspaceId") String workspaceId,
@RequestBody JobEntity entity) {
T.VerifyUtil.is(entity).notNull()
.and(entity.getEnvironmentId()).notEmpty(RCode.ENVIRONMENT_ID_CANNOT_EMPTY)
.and(entity.getPackageId()).notEmpty(RCode.PACKAGE_ID_CANNOT_EMPTY)
.and(entity.getPlaybookId()).notEmpty(RCode.PLAYBOOK_ID_CANNOT_EMPTY)
.and(entity.getWorkspaceId()).notEmpty(RCode.WORKSPACE_ID_CANNOT_EMPTY);
entity.setEnvId(entity.getEnvironmentId());
JobEntity jobEntity = jobService.saveJob(entity);
return R.ok().putData("id", jobEntity.getId());
}
@DeleteMapping("/{workspaceId}/job")
public R delete(@PathVariable("workspaceId") String workspaceId,
@RequestParam String ids) {
T.VerifyUtil.is(ids).notEmpty();
List<String> idList = Arrays.asList(ids.split(","));
jobService.removeJob(idList);
return R.ok();
}
@PutMapping("/{workspaceId}/job/cancel")
public R cancel(@PathVariable("workspaceId") String workspaceId,
@RequestParam String ids) {
T.VerifyUtil.is(ids).notEmpty();
List<String> idList = Arrays.asList(ids.split(","));
// TODO 其他处理
// update state
jobService.update(new LambdaUpdateWrapper<JobEntity>()
.in(JobEntity::getId, idList)
.set(JobEntity::getStatus, "cancel")
);
return R.ok();
}
}

View File

@@ -1,174 +0,0 @@
//package net.geedge.asw.module.runner.controller;
//
//import cn.dev33.satoken.annotation.SaIgnore;
//import cn.hutool.core.lang.Opt;
//import cn.hutool.log.Log;
//import com.baomidou.mybatisplus.core.conditions.update.LambdaUpdateWrapper;
//import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
//import jakarta.servlet.http.HttpServletResponse;
//import net.geedge.asw.common.util.R;
//import net.geedge.asw.common.util.RCode;
//import net.geedge.asw.common.util.T;
//import net.geedge.asw.module.app.entity.PackageEntity;
//import net.geedge.asw.module.runner.entity.JobEntity;
//import net.geedge.asw.module.runner.entity.PlaybookEntity;
//import net.geedge.asw.module.runner.entity.RunnerEntity;
//import net.geedge.asw.module.runner.service.IJobService;
//import net.geedge.asw.module.runner.service.IRunnerService;
//import org.springframework.beans.factory.annotation.Autowired;
//import org.springframework.web.bind.annotation.*;
//import org.springframework.web.multipart.MultipartFile;
//
//import java.io.IOException;
//import java.util.Map;
//
//@RestController
//@RequestMapping("/api/v1/runner")
//public class RunnerController {
//
// private static final Log log = Log.get();
//
// @Autowired
// private IJobService jobService;
//
// @Autowired
// private IRunnerService runnerService;
//
// @GetMapping("/{id}")
// public R detail(@PathVariable("id") String id) {
// RunnerEntity runnerEntity = runnerService.getById(id);
// return R.ok().putData("record", runnerEntity);
// }
//
// @GetMapping
// public R list(@RequestParam Map<String, Object> params) {
// T.VerifyUtil.is(params).notNull()
// .and(T.MapUtil.getStr(params, "workspaceId")).notEmpty(RCode.WORKSPACE_ID_CANNOT_EMPTY);
//
// Page page = runnerService.queryList(params);
// return R.ok(page);
// }
//
// @PostMapping
// public R add(@RequestBody RunnerEntity entity) {
// T.VerifyUtil.is(entity).notNull()
// .and(entity.getWorkspaceId()).notEmpty(RCode.WORKSPACE_ID_CANNOT_EMPTY);
//
// RunnerEntity runner = runnerService.saveRunner(entity);
// return R.ok().putData("record", runner);
// }
//
// @PutMapping
// public R update(@RequestBody RunnerEntity entity) {
// T.VerifyUtil.is(entity).notNull()
// .and(entity.getId()).notEmpty(RCode.ID_CANNOT_EMPTY)
// .and(entity.getWorkspaceId()).notEmpty(RCode.WORKSPACE_ID_CANNOT_EMPTY);
//
// RunnerEntity runner = runnerService.updateRunner(entity);
// return R.ok().putData("record", runner);
// }
//
// @DeleteMapping("/{id}")
// public R delete(@PathVariable("id") String id) {
// runnerService.removeById(id);
// return R.ok();
// }
//
// @SaIgnore
// @PostMapping("/register")
// public void register(@RequestHeader("Authorization") String token, HttpServletResponse response) throws IOException {
// RunnerEntity runner = runnerService.getOne(new LambdaUpdateWrapper<RunnerEntity>().eq(RunnerEntity::getToken, token));
// String status = Opt.ofNullable(runner).map(RunnerEntity::getStatus).orElseGet(() -> null);
// if (!T.StrUtil.equals("online", status)) {
// log.warn("[register] [runner is offline] [token: {}]", token);
// response.sendError(HttpServletResponse.SC_FORBIDDEN, "Runner is offline");
// }
// }
//
// @SaIgnore
// @PostMapping("/heartbeat")
// public void heartbeat(@RequestHeader("Authorization") String token, @RequestBody Map<String, Integer> platformMap,
// HttpServletResponse response) throws IOException {
// RunnerEntity runner = runnerService.getOne(new LambdaUpdateWrapper<RunnerEntity>().eq(RunnerEntity::getToken, token));
// String status = Opt.ofNullable(runner).map(RunnerEntity::getStatus).orElseGet(() -> null);
// if (!T.StrUtil.equals("online", status)) {
// log.warn("[heartbeat] [runner is offline] [token: {}]", token);
// response.sendError(HttpServletResponse.SC_FORBIDDEN, "Runner is offline");
// return;
// }
//
// // update last_heartbeat_timestamp
// runnerService.update(new LambdaUpdateWrapper<RunnerEntity>()
// .set(RunnerEntity::getLastHeartbeatTimestamp, System.currentTimeMillis())
// .eq(RunnerEntity::getId, runner.getId()));
//
// // findjob by platform
// String platform = platformMap.entrySet().stream().filter(entry -> entry.getValue() > 0).findFirst().map(entry -> entry.getKey()).orElseGet(null);
// JobEntity job = jobService.assignPendingJob(runner.getId(), platform);
// if (T.ObjectUtil.isNotNull(job)) {
// // package
// PackageEntity pkg = job.getPkg();
// Map<String, String> pkgInfo = T.MapUtil.builder("id", pkg.getId())
// .put("platform", pkg.getPlatform())
// .put("identifier", pkg.getIdentifier())
// .put("version", pkg.getVersion())
// .build();
//
// // playbook
// PlaybookEntity playbook = job.getPlaybook();
// Map<String, String> pbInfo = T.MapUtil.builder("id", playbook.getId())
// .put("name", playbook.getName())
// .build();
//
// // response job info
// Map<Object, Object> responseData = T.MapUtil.builder()
// .put("id", job.getId())
// .put("pkg", pkgInfo)
// .put("playbook", pbInfo)
// .build();
// response.setCharacterEncoding("UTF-8");
// response.setContentType("text/html; charset=UTF-8");
// response.getWriter().write(T.JSONUtil.toJsonStr(responseData));
// }
// }
//
// @SaIgnore
// @PutMapping("/trace/{jobId}")
// public void trace(@RequestHeader("Authorization") String token, @PathVariable String jobId, @RequestBody byte[] bytes,
// HttpServletResponse response) throws IOException {
// RunnerEntity runner = runnerService.getOne(new LambdaUpdateWrapper<RunnerEntity>().eq(RunnerEntity::getToken, token));
// String status = Opt.ofNullable(runner).map(RunnerEntity::getStatus).orElseGet(() -> null);
// if (!T.StrUtil.equals("online", status)) {
// log.warn("[trace] [runner is offline] [token: {}]", token);
// response.sendError(HttpServletResponse.SC_FORBIDDEN, "Runner is offline");
// return;
// }
//
// try {
// // 追加到文件中
// String content = T.StrUtil.str(bytes, T.CharsetUtil.CHARSET_UTF_8);
// jobService.appendTraceLogStrToFile(jobId, content);
// } catch (Exception e) {
// log.error("[trace] [error] [job: {}]", jobId);
// response.sendError(HttpServletResponse.SC_INTERNAL_SERVER_ERROR);
// }
// }
//
// @SaIgnore
// @PutMapping("/jobResult/{jobId}")
// public void jobResult(@RequestHeader("Authorization") String token, @PathVariable String jobId, @RequestParam String state,
// @RequestParam(value = "file", required = false) MultipartFile pcapFile,
// HttpServletResponse response) throws IOException {
// RunnerEntity runner = runnerService.getOne(new LambdaUpdateWrapper<RunnerEntity>().eq(RunnerEntity::getToken, token));
// String status = Opt.ofNullable(runner).map(RunnerEntity::getStatus).orElseGet(() -> null);
// if (!T.StrUtil.equals("online", status)) {
// log.warn("[trace] [runner is offline] [token: {}]", token);
// response.sendError(HttpServletResponse.SC_FORBIDDEN, "Runner is offline");
// return;
// }
//
// // 更新任务状态
// jobService.updateJobResult(jobId, state, pcapFile);
// }
//
//}

View File

@@ -1,30 +0,0 @@
package net.geedge.asw.module.runner.entity;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
import lombok.Data;
@Data
@TableName("runner")
public class RunnerEntity {
@TableId(type = IdType.ASSIGN_UUID)
private String id;
private String name;
private String token;
private String tags;
private String supportPlatforms;
private Integer shareFlag;
private String description;
private String status;
private Long lastHeartbeatTimestamp;
private Long createTimestamp;
private Long updateTimestamp;
private String createUserId;
private String updateUserId;
private String workspaceId;
}

View File

@@ -1,134 +0,0 @@
package net.geedge.asw.module.runner.job;
import cn.hutool.http.Header;
import cn.hutool.http.HttpRequest;
import cn.hutool.http.HttpResponse;
import cn.hutool.json.JSONObject;
import cn.hutool.log.Log;
import net.geedge.asw.common.util.Constants;
import net.geedge.asw.common.util.RCode;
import net.geedge.asw.common.util.T;
import net.geedge.asw.module.environment.entity.EnvironmentEntity;
import net.geedge.asw.module.environment.service.IEnvironmentService;
import net.geedge.asw.module.runner.entity.JobEntity;
import net.geedge.asw.module.runner.entity.PcapEntity;
import net.geedge.asw.module.runner.service.IJobService;
import net.geedge.asw.module.runner.service.IPcapService;
import net.geedge.asw.module.runner.util.RunnerConstant;
import org.apache.commons.lang3.time.StopWatch;
import org.quartz.DisallowConcurrentExecution;
import org.quartz.JobExecutionContext;
import org.quartz.JobExecutionException;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.core.io.FileSystemResource;
import org.springframework.core.io.Resource;
import org.springframework.http.MediaType;
import org.springframework.scheduling.quartz.QuartzJobBean;
import java.io.File;
import java.util.Map;
import java.util.Set;
@DisallowConcurrentExecution
public class JobPlaybookExecResultChecker extends QuartzJobBean {
private static final Log log = Log.get();
@Autowired
private IEnvironmentService environmentService;
@Autowired
private IPcapService pcapService;
@Autowired
private IJobService jobService;
@Override
protected void executeInternal(JobExecutionContext context) throws JobExecutionException {
Thread.currentThread().setName("JobPlaybookExecResultChecker");
log.info("[JobPlaybookExecResultChecker] [begin]");
StopWatch sw = new StopWatch();
sw.start();
try {
this.playbookExecResultChecker();
} catch (Exception e) {
log.error(e, "[JobPlaybookExecResultChecker] [error]");
} finally {
sw.stop();
}
log.info("[JobPlaybookExecResultChecker] [finshed] [Run Time: {}]", sw.toString());
}
private void playbookExecResultChecker() {
Set<Map.Entry<String, String>> entryList = Constants.PLAYBOOK_EXECUTOR_RESULT.entrySet();
if (entryList.isEmpty()) {
return;
}
for (Map.Entry<String, String> entry : entryList) {
Thread.ofVirtual().start(() -> {
String tid = entry.getKey();
String jobId = entry.getValue();
JobEntity job = jobService.getById(jobId);
EnvironmentEntity environment = environmentService.getById(job.getEnvId());
log.info("[playbookExecResultChecker] [tid: {}] [jobId: {}] [envId]", tid, jobId, environment.getId());
JSONObject paramJSONObject = environment.getParamJSONObject();
String url = paramJSONObject.getStr("url");
String token = paramJSONObject.getStr("token");
HttpRequest request = T.HttpUtil.createGet(String.format("%s/api/v1/env/playbook/%s", url, tid));
request.header("Authorization", token);
HttpResponse response = request.execute();
log.info("[playbookExecResultChecker] [env: {}] [status: {}]", environment.getId(), response.getStatus());
File destination = null;
if (response.isOk()) {
// file
if (MediaType.APPLICATION_OCTET_STREAM_VALUE.equals(response.header(Header.CONTENT_TYPE.getValue()))) {
String fileName = response.header(Header.CONTENT_DISPOSITION).split("filename=")[1];
if (log.isDebugEnabled()) {
log.debug("[playbookExecResultChecker] [env: {}] [result fileName: {}]", environment.getId(), fileName);
}
destination = T.FileUtil.file(Constants.TEMP_PATH, fileName);
T.FileUtil.writeBytes(response.bodyBytes(), destination);
Resource fileResource = new FileSystemResource(destination);
// upload pcap file
PcapEntity pcapEntity = pcapService.savePcap(fileResource, "", job.getWorkspaceId(), job.getCreateUserId());
job.setPcapId(pcapEntity.getId());
job.setStatus(RunnerConstant.JobStatus.PASSED.getValue());
job.setEndTimestamp(System.currentTimeMillis());
job.setUpdateTimestamp(System.currentTimeMillis());
jobService.updateById(job);
Constants.PLAYBOOK_EXECUTOR_RESULT.remove(tid);
} else {
String result = response.body();
if (log.isDebugEnabled()) {
log.debug("[playbookExecResultChecker] [env: {}] [result: {}]", environment.getId(), result);
}
JSONObject jsonObject = T.JSONUtil.parseObj(result);
if (T.ObjectUtil.equal(RCode.SUCCESS.getCode(), jsonObject.getInt("code"))) {
JSONObject data = jsonObject.getJSONObject("data");
String status = data.getStr("status");
if (!RunnerConstant.JobStatus.RUNNING.getValue().equals(status)) {
job.setStatus(RunnerConstant.JobStatus.FAILED.getValue());
job.setUpdateTimestamp(System.currentTimeMillis());
job.setEndTimestamp(System.currentTimeMillis());
jobService.updateById(job);
Constants.PLAYBOOK_EXECUTOR_RESULT.remove(tid);
}
}
}
}
if (destination != null) {
T.FileUtil.del(destination);
}
});
}
}
}

View File

@@ -1,143 +0,0 @@
package net.geedge.asw.module.runner.job;
import cn.hutool.http.HttpRequest;
import cn.hutool.http.HttpResponse;
import cn.hutool.json.JSONObject;
import cn.hutool.log.Log;
import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
import com.baomidou.mybatisplus.core.conditions.update.LambdaUpdateWrapper;
import net.geedge.asw.common.util.Constants;
import net.geedge.asw.common.util.RCode;
import net.geedge.asw.common.util.T;
import net.geedge.asw.module.app.entity.PackageEntity;
import net.geedge.asw.module.app.service.IPackageService;
import net.geedge.asw.module.environment.entity.EnvironmentEntity;
import net.geedge.asw.module.environment.service.IEnvironmentService;
import net.geedge.asw.module.runner.entity.JobEntity;
import net.geedge.asw.module.runner.entity.PlaybookEntity;
import net.geedge.asw.module.runner.service.IJobService;
import net.geedge.asw.module.runner.service.IPlaybookService;
import net.geedge.asw.module.runner.util.RunnerConstant;
import org.apache.commons.lang3.time.StopWatch;
import org.quartz.DisallowConcurrentExecution;
import org.quartz.JobExecutionContext;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.scheduling.quartz.QuartzJobBean;
import org.springframework.transaction.annotation.Transactional;
import java.io.File;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
@DisallowConcurrentExecution
public class JobPlaybookExecutor extends QuartzJobBean {
private static final Log log = Log.get();
@Autowired
private IJobService jobService;
@Autowired
private IEnvironmentService environmentService;
@Autowired
private IPackageService packageService;
@Autowired
private IPlaybookService playbookService;
@Override
protected void executeInternal(JobExecutionContext context) {
Thread.currentThread().setName("JobPlaybookExecutor");
log.info("[JobPlaybookExecutor] [begin]");
StopWatch sw = new StopWatch();
sw.start();
try {
this.playbookExecutor();
} catch (Exception e) {
log.error(e, "[JobPlaybookExecutor] [error]");
} finally {
sw.stop();
}
log.info("[JobPlaybookExecutor] [finshed] [Run Time: {}]", sw.toString());
}
@Transactional(rollbackFor = Exception.class)
public void playbookExecutor() {
List<JobEntity> list = jobService.list(new LambdaQueryWrapper<JobEntity>().eq(JobEntity::getStatus, RunnerConstant.JobStatus.CREATED.getValue()));
Map<String, List<JobEntity>> jobByEnvList = list.stream().collect(Collectors.groupingBy(JobEntity::getEnvId));
for (Map.Entry<String, List<JobEntity>> jobByEnv : jobByEnvList.entrySet()) {
String envId = jobByEnv.getKey();
List<JobEntity> jobList = jobByEnv.getValue();
Thread.ofVirtual().start(() -> {
for (JobEntity job : jobList) {
List<JobEntity> JobRunList = jobService.list(new LambdaQueryWrapper<JobEntity>().eq(JobEntity::getStatus, RunnerConstant.JobStatus.RUNNING.getValue()).eq(JobEntity::getEnvId, envId));
if (T.CollUtil.isNotEmpty(JobRunList)) {
continue;
}
EnvironmentEntity environment = environmentService.getById(envId);
if (!environment.getStatus().equals(1)) {
if (log.isDebugEnabled()) {
log.debug("[playbookExecutor] [environment is not available] [jobId: {}] [envId: {}]", job.getId(), environment.getId());
}
continue;
}
String result = null;
String playbookId = job.getPlaybookId();
String packageId = job.getPackageId();
PackageEntity packageEntity = packageService.getById(packageId);
File packageFile = T.FileUtil.file(packageEntity.getPath());
String packageName = packageEntity.getIdentifier();
PlaybookEntity playbook = playbookService.getById(playbookId);
File playbookFile = T.FileUtil.file(playbook.getPath());
log.info("[playbookExecutor] [jobId: {}] [envId: {}] [playbookId: {}] [packageId: {}]", job.getId(), environment.getId(), playbookId, packageId);
JSONObject paramJSONObject = environment.getParamJSONObject();
String url = paramJSONObject.getStr("url");
String token = paramJSONObject.getStr("token");
HttpRequest request = T.HttpUtil.createPost(String.format("%s/api/v1/env/playbook", url));
request.form("files", packageFile, playbookFile);
request.form("packageName", packageName);
request.header("Authorization", token);
HttpResponse response = request.execute();
log.info("[playbookExecutor] [env] [status: {}]", environment.getId(), response.getStatus());
if (response.isOk()) {
result = response.body();
}
if (log.isDebugEnabled()) {
log.debug("[playbookExecutor] [env: {}] [result: {}]", environment.getId(), result);
}
if (T.StrUtil.isNotEmpty(result)) {
try {
JSONObject jsonObject = T.JSONUtil.parseObj(result);
if (T.ObjectUtil.equal(RCode.SUCCESS.getCode(), jsonObject.getInt("code"))) {
JSONObject data = jsonObject.getJSONObject("data");
String tid = data.getStr("tid");
Constants.PLAYBOOK_EXECUTOR_RESULT.put(tid, job.getId());
}
} catch (Exception e) {
log.error(e, "[playbookExecutor] [parse result error] [result: {}]", job.getId(), result);
}
}
// update job status, starTime, updateTimestamp
jobService.update(new LambdaUpdateWrapper<JobEntity>()
.set(JobEntity::getStatus, RunnerConstant.JobStatus.RUNNING.getValue())
.set(JobEntity::getUpdateTimestamp, System.currentTimeMillis())
.set(JobEntity::getStartTimestamp, System.currentTimeMillis())
.eq(JobEntity::getId, job.getId())
);
}
});
}
}
}

View File

@@ -1,26 +0,0 @@
package net.geedge.asw.module.runner.service;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import com.baomidou.mybatisplus.extension.service.IService;
import net.geedge.asw.module.runner.entity.JobEntity;
import java.util.List;
import java.util.Map;
public interface IJobService extends IService<JobEntity>{
JobEntity queryInfo(String id);
Page queryList(Map<String, Object> params);
JobEntity saveJob(JobEntity entity);
void removeJob(List<String> ids);
// JobEntity assignPendingJob(String id, String platform);
//
// void appendTraceLogStrToFile(String jobId, String content) throws RuntimeException;
//
// void updateJobResult(String jobId, String state, MultipartFile pcapFile);
}

View File

@@ -1,17 +0,0 @@
//package net.geedge.asw.module.runner.service;
//
//import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
//import com.baomidou.mybatisplus.extension.service.IService;
//import net.geedge.asw.module.runner.entity.RunnerEntity;
//
//import java.util.Map;
//
//public interface IRunnerService extends IService<RunnerEntity>{
//
// Page queryList(Map<String, Object> params);
//
// RunnerEntity saveRunner(RunnerEntity entity);
//
// RunnerEntity updateRunner(RunnerEntity entity);
//
//}

View File

@@ -1,159 +0,0 @@
package net.geedge.asw.module.runner.service.impl;
import cn.dev33.satoken.stp.StpUtil;
import cn.hutool.log.Log;
import com.baomidou.mybatisplus.core.conditions.update.LambdaUpdateWrapper;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;
import net.geedge.asw.common.util.RCode;
import net.geedge.asw.common.util.T;
import net.geedge.asw.module.app.entity.PackageEntity;
import net.geedge.asw.module.app.service.IPackageService;
import net.geedge.asw.module.environment.entity.EnvironmentEntity;
import net.geedge.asw.module.environment.service.IEnvironmentService;
import net.geedge.asw.module.runner.dao.JobDao;
import net.geedge.asw.module.runner.entity.JobEntity;
import net.geedge.asw.module.runner.entity.PlaybookEntity;
import net.geedge.asw.module.runner.service.IJobService;
import net.geedge.asw.module.runner.service.IPlaybookService;
import net.geedge.asw.module.runner.util.RunnerConstant;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
import java.io.File;
import java.util.List;
import java.util.Map;
@Service
public class JobServiceImpl extends ServiceImpl<JobDao, JobEntity> implements IJobService {
private static final Log log = Log.get();
@Autowired
private IEnvironmentService environmentService;
@Autowired
private IPlaybookService playbookService;
@Autowired
private IPackageService packageService;
/**
* rootPath/result/{jobId}
*
* @param jobId
* @return
*/
private String getJobResultPath(String jobId) {
return T.FileUtil.file(T.WebPathUtil.getRootPath(), "result", jobId).getPath();
}
@Override
public JobEntity queryInfo(String id) {
JobEntity job = this.getById(id);
T.VerifyUtil.is(job).notNull(RCode.SYS_RECORD_NOT_FOUND);
EnvironmentEntity env = environmentService.getById(job.getEnvId());
job.setEnvironment(env);
PlaybookEntity playbook = playbookService.getById(job.getPlaybookId());
job.setPlaybook(playbook);
PackageEntity pkg = packageService.getById(job.getPackageId());
job.setPkg(pkg);
return job;
}
@Override
public Page queryList(Map<String, Object> params) {
Page page = T.PageUtil.getPage(params);
List<JobEntity> jobList = this.getBaseMapper().queryList(page, params);
page.setRecords(jobList);
return page;
}
@Override
@Transactional(rollbackFor = Exception.class)
public JobEntity saveJob(JobEntity entity) {
entity.setCreateTimestamp(System.currentTimeMillis());
entity.setUpdateTimestamp(System.currentTimeMillis());
entity.setCreateUserId(StpUtil.getLoginIdAsString());
entity.setUpdateUserId(StpUtil.getLoginIdAsString());
entity.setStatus(RunnerConstant.JobStatus.CREATED.getValue());
// save
this.save(entity);
// trace log file path
File traceLogFile = T.FileUtil.file(this.getJobResultPath(entity.getId()), "trace.log");
this.update(new LambdaUpdateWrapper<JobEntity>()
.set(JobEntity::getLogPath, traceLogFile.getPath())
.eq(JobEntity::getId, entity.getId()));
return entity;
}
@Override
@Transactional(rollbackFor = Exception.class)
public void removeJob(List<String> ids) {
// remove
this.removeBatchByIds(ids);
}
// @Override
// public synchronized JobEntity assignPendingJob(String runnerId, String platform) {
// if (T.StrUtil.hasEmpty(runnerId, platform)) {
// return null;
// }
// // query
// JobEntity job = this.getBaseMapper().getPendingJobByPlatform(platform);
// if (T.ObjectUtil.isNotNull(job)) {
// // update
// this.update(new LambdaUpdateWrapper<JobEntity>()
// .set(JobEntity::getRunnerId, runnerId)
// .set(JobEntity::getStatus, RunnerConstant.JobStatus.RUNNING.getValue())
// .set(JobEntity::getStartTimestamp, System.currentTimeMillis())
// .eq(JobEntity::getId, job.getId())
// );
// }
// return job;
// }
//
// @Override
// public void appendTraceLogStrToFile(String jobId, String content) throws RuntimeException {
// try {
// JobEntity job = this.getById(jobId);
// if (T.StrUtil.isEmpty(job.getLogPath())) {
// File traceLogFile = T.FileUtil.file(this.getJobResultPath(jobId), "trace.log");
// job.setLogPath(traceLogFile.getPath());
// }
// // append content
// T.FileUtil.appendString(content, T.FileUtil.file(job.getLogPath()), T.CharsetUtil.CHARSET_UTF_8);
// } catch (IORuntimeException e) {
// log.error(e, "[appendTraceLogStrToFile] [error] [job: {}] [content: {}]", jobId, content);
// throw new RuntimeException(e.getMessage());
// }
// }
//
// @Override
// @Transactional(rollbackFor = Exception.class)
// public void updateJobResult(String jobId, String state, MultipartFile pcapFile) {
// String pcapId = T.StrUtil.EMPTY;
// // save pcap file
// if (T.ObjectUtil.isNotNull(pcapFile)) {
// PcapEntity pcapEntity = pcapService.savePcap(jobId, pcapFile.getResource());
// pcapId = pcapEntity.getId();
// }
//
// // update job status&pcap_id
// state = T.StrUtil.equals("success", state) ? RunnerConstant.JobStatus.PASSED.getValue() : state;
// this.update(new LambdaUpdateWrapper<JobEntity>()
// .set(JobEntity::getStatus, state)
// .set(T.StrUtil.isNotEmpty(pcapId), JobEntity::getPcapId, pcapId)
// .set(JobEntity::getEndTimestamp, System.currentTimeMillis())
// .eq(JobEntity::getId, jobId)
// );
// }
}

View File

@@ -1,84 +0,0 @@
package net.geedge.asw.module.runner.service.impl;
import cn.dev33.satoken.stp.StpUtil;
import cn.hutool.log.Log;
import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;
import net.geedge.asw.common.config.Query;
import net.geedge.asw.common.util.*;
import net.geedge.asw.module.runner.dao.PlaybookDao;
import net.geedge.asw.module.runner.entity.PlaybookEntity;
import net.geedge.asw.module.runner.service.IPlaybookService;
import org.apache.commons.io.FileUtils;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
import org.springframework.web.multipart.MultipartFile;
import java.io.File;
import java.util.Arrays;
import java.util.List;
import java.util.Map;
@Service
public class PlaybookServiceImpl extends ServiceImpl<PlaybookDao, PlaybookEntity> implements IPlaybookService {
private final static Log log = Log.get();
@Override
public PlaybookEntity detail(String workspaceId, String id) {
PlaybookEntity playbook = this.baseMapper.queryInfo(workspaceId, id);
return playbook;
}
@Override
public Page queryList(String workspaceId, Map params) {
Page page = new Query(PlaybookEntity.class).getPage(params);
List<PlaybookEntity> playbookList = this.baseMapper.queryList(page, workspaceId, params);
page.setRecords(playbookList);
return page;
}
@Override
@Transactional(rollbackFor = Exception.class)
public PlaybookEntity savePlaybook(String workspaceId, MultipartFile file, String name, String description) {
List<PlaybookEntity> playbookList = this.baseMapper.selectList(new LambdaQueryWrapper<PlaybookEntity>().eq(PlaybookEntity::getWorkspaceId, workspaceId).eq(PlaybookEntity::getName, name));
if (T.CollUtil.isNotEmpty(playbookList)) {
throw new ASWException(RCode.PLAYBOOK_NAME_DUPLICATE);
}
PlaybookEntity playbook = new PlaybookEntity();
try {
playbook.setWorkspaceId(workspaceId);
playbook.setName(name);
playbook.setDescription(description);
playbook.setCreateUserId(StpUtil.getLoginIdAsString());
playbook.setUpdateUserId(StpUtil.getLoginIdAsString());
playbook.setCreateTimestamp(System.currentTimeMillis());
playbook.setUpdateTimestamp(System.currentTimeMillis());
// path
File destination = T.FileUtil.file(Constants.PLAYBOOK_FILES_DIR, name);
FileUtils.copyInputStreamToFile(file.getInputStream(), destination);
playbook.setPath(destination.getPath());
this.save(playbook);
} catch (Exception e) {
log.error(e, "[savePlaybook] [error] [file: {}]", file.getName());
T.FileUtil.del(description);
}
return playbook;
}
@Override
public void delete(String workspaceId, String ids) {
List<String> idList = Arrays.asList(ids.split(","));
for (String id : idList) {
PlaybookEntity entity = this.getById(id);
// remove file
T.FileUtil.del(entity.getPath());
// remove
this.removeById(id);
}
}
}

View File

@@ -1,51 +0,0 @@
//package net.geedge.asw.module.runner.service.impl;
//
//import cn.dev33.satoken.stp.StpUtil;
//import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
//import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;
//import net.geedge.asw.common.util.T;
//import net.geedge.asw.module.runner.dao.RunnerDao;
//import net.geedge.asw.module.runner.entity.RunnerEntity;
//import net.geedge.asw.module.runner.service.IRunnerService;
//import org.springframework.stereotype.Service;
//
//import java.util.List;
//import java.util.Map;
//
//@Service
//public class RunnerServiceImpl extends ServiceImpl<RunnerDao, RunnerEntity> implements IRunnerService {
//
// @Override
// public Page queryList(Map<String, Object> params) {
// Page page = T.PageUtil.getPage(params);
// List<RunnerEntity> jobList = this.getBaseMapper().queryList(page, params);
// page.setRecords(jobList);
// return page;
// }
//
// @Override
// public RunnerEntity saveRunner(RunnerEntity entity) {
// entity.setCreateTimestamp(System.currentTimeMillis());
// entity.setUpdateTimestamp(System.currentTimeMillis());
// entity.setCreateUserId(StpUtil.getLoginIdAsString());
// entity.setUpdateUserId(StpUtil.getLoginIdAsString());
//
// // token
// entity.setToken(T.IdUtil.fastSimpleUUID());
//
// // save
// this.save(entity);
// return entity;
// }
//
// @Override
// public RunnerEntity updateRunner(RunnerEntity entity) {
// entity.setUpdateTimestamp(System.currentTimeMillis());
// entity.setUpdateUserId(StpUtil.getLoginIdAsString());
//
// // update
// this.updateById(entity);
// return entity;
// }
//
//}

View File

@@ -14,9 +14,7 @@ import net.geedge.asw.module.sys.service.ISysAuthService;
import org.springframework.beans.factory.annotation.Autowired; import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.*; import org.springframework.web.bind.annotation.*;
import java.util.List;
import java.util.Map; import java.util.Map;
import java.util.stream.Collectors;
@RestController @RestController
@RequestMapping("/api/v1") @RequestMapping("/api/v1")
@@ -44,12 +42,13 @@ public class SysAuthController {
// payload // payload
jwt.setPayload("sub", userEntity.getUserName()); jwt.setPayload("sub", userEntity.getUserName());
Map<String, Object> permissions = authService.userPermissions(); // Map<String, Object> permissions = authService.userPermissions();
//
String roles = ((List<String>) T.JSONUtil.getByPath(T.JSONUtil.parse(permissions), "records.role.name")).stream() // String roles = ((List<String>) T.JSONUtil.getByPath(T.JSONUtil.parse(permissions), "records.role.name")).stream()
.distinct() // .distinct()
.collect(Collectors.joining(",")); // .collect(Collectors.joining(","));
jwt.setPayload("roles", roles); // TODO asw and opensearch role mapping
jwt.setPayload("roles", "admin");
Long eff = Long.valueOf(jwt.getPayload("eff").toString()); Long eff = Long.valueOf(jwt.getPayload("eff").toString());
jwt.setPayload("exp", eff == -1L ? -1 : (eff / 1000)); jwt.setPayload("exp", eff == -1L ? -1 : (eff / 1000));

View File

@@ -4,6 +4,7 @@ import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableField; import com.baomidou.mybatisplus.annotation.TableField;
import com.baomidou.mybatisplus.annotation.TableId; import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName; import com.baomidou.mybatisplus.annotation.TableName;
import com.fasterxml.jackson.annotation.JsonIgnore;
import lombok.Data; import lombok.Data;
import net.geedge.asw.module.workspace.entity.WorkspaceMemberEntity; import net.geedge.asw.module.workspace.entity.WorkspaceMemberEntity;
@@ -21,6 +22,7 @@ public class SysUserEntity {
@TableField("user_name") @TableField("user_name")
private String userName; private String userName;
@JsonIgnore
private String pwd; private String pwd;
@TableField(exist = false) @TableField(exist = false)

View File

@@ -44,6 +44,9 @@ public class SysAuthServiceImpl implements ISysAuthService {
@Override @Override
public SysUserEntity login(String userName, String pwd) { public SysUserEntity login(String userName, String pwd) {
if (T.StrUtil.equals(userName, "system")) {
throw ASWException.builder().rcode(RCode.SYS_SYSTEM_USER_NOT_LOGIN).build();
}
SysUserEntity userEntity = userDao SysUserEntity userEntity = userDao
.selectOne(new QueryWrapper<SysUserEntity>().lambda().eq(SysUserEntity::getUserName, userName)); .selectOne(new QueryWrapper<SysUserEntity>().lambda().eq(SysUserEntity::getUserName, userName));
if (T.ObjectUtil.isNull(userEntity) if (T.ObjectUtil.isNull(userEntity)

View File

@@ -81,7 +81,7 @@ public class SysRoleServiceImpl extends ServiceImpl<SysRoleDao, SysRoleEntity> i
@Override @Override
public Page queryList(Map params) { public Page queryList(Map params) {
params.put("orderBy", T.StrUtil.emptyToDefault(T.MapUtil.getStr(params, "orderBy"), "name")); params.put("orderBy", T.StrUtil.emptyToDefault(T.MapUtil.getStr(params, "orderBy"), "create_timestamp"));
Page page = new Query(SysRoleEntity.class).getPage(params); Page page = new Query(SysRoleEntity.class).getPage(params);
List<SysRoleEntity> roleList = this.getBaseMapper().queryList(page, params); List<SysRoleEntity> roleList = this.getBaseMapper().queryList(page, params);
page.setRecords(roleList); page.setRecords(roleList);

View File

@@ -5,6 +5,7 @@ import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
import com.baomidou.mybatisplus.core.conditions.query.QueryWrapper; import com.baomidou.mybatisplus.core.conditions.query.QueryWrapper;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page; import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl; import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;
import net.geedge.asw.common.config.Query;
import net.geedge.asw.common.util.ASWException; import net.geedge.asw.common.util.ASWException;
import net.geedge.asw.common.util.Constants; import net.geedge.asw.common.util.Constants;
import net.geedge.asw.common.util.RCode; import net.geedge.asw.common.util.RCode;
@@ -40,8 +41,7 @@ public class SysUserServiceImpl extends ServiceImpl<SysUserDao, SysUserEntity> i
@Override @Override
public Page<SysUserEntity> queryList(Map params) { public Page<SysUserEntity> queryList(Map params) {
Page page = new Query(SysUserEntity.class).getPage(params);
Page page = T.PageUtil.getPage(params);
List<SysUserEntity> jobList = this.getBaseMapper().queryList(page, params); List<SysUserEntity> jobList = this.getBaseMapper().queryList(page, params);
page.setRecords(jobList); page.setRecords(jobList);
return page; return page;

View File

@@ -60,7 +60,6 @@ public class WorkspaceController {
public R save(@RequestBody WorkspaceEntity workspace) { public R save(@RequestBody WorkspaceEntity workspace) {
VerifyUtil.is(workspace).notNull() VerifyUtil.is(workspace).notNull()
.and(workspace.getName()).notEmpty(RCode.WORKSPACE_NAME_CANNOT_EMPTY) .and(workspace.getName()).notEmpty(RCode.WORKSPACE_NAME_CANNOT_EMPTY)
.and(workspace.getMembers()).notEmpty(RCode.WORKSPACE_MEMBER_CANNOT_EMPTY)
.and(workspace.getVisibility()).notEmpty(RCode.WORKSPACE_VISIBILITY_CANNOT_EMPTY); .and(workspace.getVisibility()).notEmpty(RCode.WORKSPACE_VISIBILITY_CANNOT_EMPTY);
WorkspaceEntity workspaceEntity = workspaceService.saveWorkspace(workspace); WorkspaceEntity workspaceEntity = workspaceService.saveWorkspace(workspace);
@@ -73,7 +72,6 @@ public class WorkspaceController {
VerifyUtil.is(workspace).notNull() VerifyUtil.is(workspace).notNull()
.and(workspace.getId()).notEmpty(RCode.WORKSPACE_ID_CANNOT_EMPTY) .and(workspace.getId()).notEmpty(RCode.WORKSPACE_ID_CANNOT_EMPTY)
.and(workspace.getName()).notEmpty(RCode.WORKSPACE_NAME_CANNOT_EMPTY) .and(workspace.getName()).notEmpty(RCode.WORKSPACE_NAME_CANNOT_EMPTY)
.and(workspace.getMembers()).notEmpty(RCode.WORKSPACE_MEMBER_CANNOT_EMPTY)
.and(workspace.getVisibility()).notEmpty(RCode.WORKSPACE_VISIBILITY_CANNOT_EMPTY); .and(workspace.getVisibility()).notEmpty(RCode.WORKSPACE_VISIBILITY_CANNOT_EMPTY);
WorkspaceEntity workspaceEntity = workspaceService.updateWorkspace(workspace); WorkspaceEntity workspaceEntity = workspaceService.updateWorkspace(workspace);

View File

@@ -5,9 +5,10 @@ import com.baomidou.mybatisplus.annotation.TableField;
import com.baomidou.mybatisplus.annotation.TableId; import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName; import com.baomidou.mybatisplus.annotation.TableName;
import lombok.Data; import lombok.Data;
import net.geedge.asw.module.sys.entity.SysRoleEntity; import net.geedge.asw.common.util.T;
import net.geedge.asw.module.sys.entity.SysUserEntity; import net.geedge.asw.module.sys.entity.SysUserEntity;
import java.io.File;
import java.util.List; import java.util.List;
@Data @Data
@@ -36,4 +37,8 @@ public class WorkspaceEntity {
@TableField(exist = false) @TableField(exist = false)
private List<WorkspaceMemberEntity> members; private List<WorkspaceMemberEntity> members;
public File buildGitPath() {
return T.FileUtil.file(T.WebPathUtil.getRootPath(), "workspace", this.getId());
}
} }

View File

@@ -4,6 +4,7 @@ import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import com.baomidou.mybatisplus.extension.service.IService; import com.baomidou.mybatisplus.extension.service.IService;
import net.geedge.asw.module.workspace.entity.WorkspaceEntity; import net.geedge.asw.module.workspace.entity.WorkspaceEntity;
import java.io.File;
import java.util.Map; import java.util.Map;
public interface IWorkspaceService extends IService<WorkspaceEntity>{ public interface IWorkspaceService extends IService<WorkspaceEntity>{
@@ -15,4 +16,7 @@ public interface IWorkspaceService extends IService<WorkspaceEntity>{
WorkspaceEntity updateWorkspace(WorkspaceEntity workspace); WorkspaceEntity updateWorkspace(WorkspaceEntity workspace);
void deleteWorkspace(String ids); void deleteWorkspace(String ids);
File getGitDir(String workspaceId);
} }

View File

@@ -9,13 +9,17 @@ import com.alibaba.fastjson2.JSONObject;
import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper; import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page; import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl; import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;
import net.geedge.asw.common.config.Query;
import net.geedge.asw.common.util.ASWException; import net.geedge.asw.common.util.ASWException;
import net.geedge.asw.common.util.Constants; import net.geedge.asw.common.util.Constants;
import net.geedge.asw.common.util.RCode; import net.geedge.asw.common.util.RCode;
import net.geedge.asw.common.util.T; import net.geedge.asw.common.util.T;
import net.geedge.asw.module.app.util.JGitUtils;
import net.geedge.asw.module.feign.client.DashboardClient; import net.geedge.asw.module.feign.client.DashboardClient;
import net.geedge.asw.module.feign.client.KibanaClient; import net.geedge.asw.module.feign.client.KibanaClient;
import net.geedge.asw.module.sys.entity.SysUserEntity;
import net.geedge.asw.module.sys.service.ISysConfigService; import net.geedge.asw.module.sys.service.ISysConfigService;
import net.geedge.asw.module.sys.service.ISysUserService;
import net.geedge.asw.module.workspace.dao.WorkspaceDao; import net.geedge.asw.module.workspace.dao.WorkspaceDao;
import net.geedge.asw.module.workspace.entity.WorkspaceEntity; import net.geedge.asw.module.workspace.entity.WorkspaceEntity;
import net.geedge.asw.module.workspace.entity.WorkspaceMemberEntity; import net.geedge.asw.module.workspace.entity.WorkspaceMemberEntity;
@@ -33,7 +37,7 @@ import java.util.Map;
@Service @Service
public class WorkspaceServiceImpl extends ServiceImpl<WorkspaceDao, WorkspaceEntity> implements IWorkspaceService { public class WorkspaceServiceImpl extends ServiceImpl<WorkspaceDao, WorkspaceEntity> implements IWorkspaceService {
private static final Log log = Log.get(); private final static Log log = Log.get();
@Autowired @Autowired
private IWorkspaceService workspaceService; private IWorkspaceService workspaceService;
@@ -50,9 +54,12 @@ public class WorkspaceServiceImpl extends ServiceImpl<WorkspaceDao, WorkspaceEnt
@Autowired @Autowired
private ISysConfigService sysConfigService; private ISysConfigService sysConfigService;
@Autowired
private ISysUserService userService;
@Override @Override
public Page queryList(Map<String, Object> params) { public Page queryList(Map<String, Object> params) {
Page page = T.PageUtil.getPage(params); Page page = new Query(WorkspaceEntity.class).getPage(params);
List<WorkspaceEntity> workspaceEntityList = this.getBaseMapper().queryList(page, params); List<WorkspaceEntity> workspaceEntityList = this.getBaseMapper().queryList(page, params);
page.setRecords(workspaceEntityList); page.setRecords(workspaceEntityList);
return page; return page;
@@ -72,13 +79,27 @@ public class WorkspaceServiceImpl extends ServiceImpl<WorkspaceDao, WorkspaceEnt
String id = workspace.getId(); String id = workspace.getId();
List<WorkspaceMemberEntity> members = workspace.getMembers(); List<WorkspaceMemberEntity> members = workspace.getMembers();
if (T.CollUtil.isNotEmpty(members)) {
members.stream().forEach(x -> { members.stream().forEach(x -> {
x.setWorkspaceId(id); x.setWorkspaceId(id);
x.setCreateTimestamp(System.currentTimeMillis()); x.setCreateTimestamp(System.currentTimeMillis());
x.setCreateUserId(StpUtil.getLoginIdAsString()); x.setCreateUserId(StpUtil.getLoginIdAsString());
}); });
workspaceMemberService.saveBatch(members); workspaceMemberService.saveBatch(members);
} else {
// workspace 默认分配 Owner 角色
WorkspaceMemberEntity member = new WorkspaceMemberEntity();
member.setWorkspaceId(workspace.getId());
member.setUserId(StpUtil.getLoginIdAsString());
member.setRoleId(Constants.BuiltInRoleEnum.OWNER.getId());
member.setCreateTimestamp(System.currentTimeMillis());
member.setCreateUserId(StpUtil.getLoginIdAsString());
workspaceMemberService.save(member);
}
// init repository
SysUserEntity loginUserEntity = userService.getById(StpUtil.getLoginIdAsString());
JGitUtils.initRepository(workspace.buildGitPath(), loginUserEntity.getName());
return workspace; return workspace;
} }
@@ -88,7 +109,6 @@ public class WorkspaceServiceImpl extends ServiceImpl<WorkspaceDao, WorkspaceEnt
try { try {
// index name // index name
String indexName = String.format("workspace-%s-*", workspace.getName()); String indexName = String.format("workspace-%s-*", workspace.getName());
String indexId = T.StrUtil.uuid();
SaTokenInfo tokenInfo = StpUtil.getTokenInfo(); SaTokenInfo tokenInfo = StpUtil.getTokenInfo();
String token = tokenInfo.getTokenValue(); String token = tokenInfo.getTokenValue();
@@ -110,7 +130,7 @@ public class WorkspaceServiceImpl extends ServiceImpl<WorkspaceDao, WorkspaceEnt
// delete index-patten // delete index-patten
if (indexExists) { if (indexExists) {
indexId = savedObjects.stream().map(x -> { String indexId = savedObjects.stream().map(x -> {
return ((JSONObject) x).getString("id"); return ((JSONObject) x).getString("id");
}).findFirst().get(); }).findFirst().get();
kibanaClient.deleteIndexPattern(token, indexId, true); kibanaClient.deleteIndexPattern(token, indexId, true);
@@ -121,9 +141,8 @@ public class WorkspaceServiceImpl extends ServiceImpl<WorkspaceDao, WorkspaceEnt
} }
// dashboard Name // dashboard Name
String dashboardName = String.format("workspace-%s", workspace.getName()); String dashboardName = String.format("dashboard-%s", workspace.getName());
Map<Object, Object> params = T.MapUtil.builder() Map<Object, Object> params = T.MapUtil.builder()
.put("indexId", indexId)
.put("indexName", indexName) .put("indexName", indexName)
.put("dashboardName", dashboardName) .put("dashboardName", dashboardName)
.build(); .build();
@@ -141,14 +160,21 @@ public class WorkspaceServiceImpl extends ServiceImpl<WorkspaceDao, WorkspaceEnt
// get dashboardId // get dashboardId
JSONObject dashboardObj = kibanaClient.findIndexPattern(token, "dashboard", dashboardName); JSONObject dashboardObj = kibanaClient.findIndexPattern(token, "dashboard", dashboardName);
savedObjects = dashboardObj.getJSONArray("saved_objects"); savedObjects = dashboardObj.getJSONArray("saved_objects");
String dashboardId = savedObjects.stream().map(x -> { String dashboardId = savedObjects.stream()
.filter(x -> {
JSONObject attributes = ((JSONObject) x).getJSONObject("attributes");
if (T.ObjectUtil.isEmpty(attributes)) return false;
String title = attributes.getString("title");
return T.StrUtil.equals(dashboardName, title);
})
.map(x -> {
return ((JSONObject) x).getString("id"); return ((JSONObject) x).getString("id");
}).findFirst().get(); }).findFirst().get();
// get indexId // get indexId
JSONObject indexObj = kibanaClient.findIndexPattern(token, "index-pattern", indexName); JSONObject indexObj = kibanaClient.findIndexPattern(token, "index-pattern", indexName);
savedObjects = indexObj.getJSONArray("saved_objects"); savedObjects = indexObj.getJSONArray("saved_objects");
indexId = savedObjects.stream().map(x -> { String indexId = savedObjects.stream().map(x -> {
return ((JSONObject) x).getString("id"); return ((JSONObject) x).getString("id");
}).findFirst().get(); }).findFirst().get();
@@ -176,14 +202,17 @@ public class WorkspaceServiceImpl extends ServiceImpl<WorkspaceDao, WorkspaceEnt
workspace.setUpdateTimestamp(System.currentTimeMillis()); workspace.setUpdateTimestamp(System.currentTimeMillis());
workspaceService.updateById(workspace); workspaceService.updateById(workspace);
workspaceMemberService.remove(new LambdaQueryWrapper<WorkspaceMemberEntity>().eq(WorkspaceMemberEntity::getWorkspaceId, workspace.getId()));
List<WorkspaceMemberEntity> members = workspace.getMembers(); List<WorkspaceMemberEntity> members = workspace.getMembers();
if (T.CollUtil.isNotEmpty(members)) {
members.stream().forEach(x -> { members.stream().forEach(x -> {
x.setWorkspaceId(workspace.getId()); x.setWorkspaceId(workspace.getId());
x.setCreateTimestamp(System.currentTimeMillis()); x.setCreateTimestamp(System.currentTimeMillis());
x.setCreateUserId(StpUtil.getLoginIdAsString()); x.setCreateUserId(StpUtil.getLoginIdAsString());
}); });
workspaceMemberService.remove(new LambdaQueryWrapper<WorkspaceMemberEntity>().eq(WorkspaceMemberEntity::getWorkspaceId, workspace.getId()));
workspaceMemberService.saveBatch(members); workspaceMemberService.saveBatch(members);
}
return workspace; return workspace;
} }
@@ -195,8 +224,35 @@ public class WorkspaceServiceImpl extends ServiceImpl<WorkspaceDao, WorkspaceEnt
if (workspaceIds.contains("1")) { if (workspaceIds.contains("1")) {
throw new ASWException(RCode.WORKSPACE_CANNOT_DELETE); throw new ASWException(RCode.WORKSPACE_CANNOT_DELETE);
} }
List<WorkspaceEntity> entityList = workspaceService.list(new LambdaQueryWrapper<WorkspaceEntity>().in(WorkspaceEntity::getId, workspaceIds));
if (T.CollUtil.isNotEmpty(entityList)) {
workspaceService.removeBatchByIds(workspaceIds); workspaceService.removeBatchByIds(workspaceIds);
workspaceMemberService.remove(new LambdaQueryWrapper<WorkspaceMemberEntity>().in(WorkspaceMemberEntity::getWorkspaceId, workspaceIds)); workspaceMemberService.remove(new LambdaQueryWrapper<WorkspaceMemberEntity>().in(WorkspaceMemberEntity::getWorkspaceId, workspaceIds));
// remove git repo
for (WorkspaceEntity entity : entityList) {
T.FileUtil.del(T.FileUtil.file(T.WebPathUtil.getRootPath(), "workspace", entity.getId()));
}
}
}
/**
* get repository path
* path= {webRootPath}/workspace/{workspace.id}
*/
@Override
public File getGitDir(String workspaceId) {
WorkspaceEntity workspace = workspaceService.getById(workspaceId);
File gitDir = workspace.buildGitPath();
// 目录不存在,初始化裸仓库
if (!gitDir.exists()) {
log.info("[getGitDir] [dir not exist] [init new repository] [path: {}]", gitDir);
SysUserEntity loginUserEntity = userService.getById(StpUtil.getLoginIdAsString());
JGitUtils.initRepository(gitDir, loginUserEntity.getName());
}
return gitDir;
} }
private void validateWorkspaceInfo(WorkspaceEntity workspace, boolean isUpdate) { private void validateWorkspaceInfo(WorkspaceEntity workspace, boolean isUpdate) {

View File

@@ -11,7 +11,7 @@ spring:
# max-age: 30d # max-age: 30d
# cache-public: true # cache-public: true
jackson: jackson:
default-property-inclusion: non_empty default-property-inclusion: NON_NULL
profiles: profiles:
active: prod active: prod
include: magic-api include: magic-api
@@ -74,3 +74,33 @@ mybatis-plus:
logging: logging:
config: ./config/logback-spring.xml config: ./config/logback-spring.xml
asw:
application:
template:
meta.json: |- #配置JSON格式
{
"id": "{uuid}",
"name": "{application name}",
"longName": "{application name}",
"developer": "",
"website": "",
"packageName": {
"android": "",
"ios": ""
},
"properties": {
"category": "",
"subcategory": "",
"content": "unknown",
"parentApp": "",
"risk": 1
}
}
signature.json: |-
{
"surrogates": []
}

View File

@@ -1,67 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd">
<mapper namespace="net.geedge.asw.module.app.dao.ApplicationDao">
<resultMap id="appResult" type="net.geedge.asw.module.app.entity.ApplicationEntity">
<result property="id" column="id"/>
<result property="name" column="name"/>
<result property="tags" column="tags"/>
<result property="packageName" column="package_name"/>
<result property="website" column="website"/>
<result property="provider" column="provider"/>
<result property="status" column="status"/>
<result property="properties" column="properties" typeHandler="com.baomidou.mybatisplus.extension.handlers.JacksonTypeHandler"/>
<result property="description" column="description"/>
<result property="createTimestamp" column="create_timestamp"/>
<result property="updateTimestamp" column="update_timestamp"/>
<result property="createUserId" column="create_user_id"/>
<result property="updateUserId" column="update_user_id"/>
<result property="workspaceId" column="workspace_id"/>
<result property="opVersion" column="op_version"/>
<association property="createUser" columnPrefix="cu_" javaType="net.geedge.asw.module.sys.entity.SysUserEntity">
<id property="id" column="id"/>
<result property="name" column="name"/>
<result property="userName" column="user_name"/>
</association>
<association property="updateUser" columnPrefix="uu_" javaType="net.geedge.asw.module.sys.entity.SysUserEntity">
<id property="id" column="id"/>
<result property="name" column="name"/>
<result property="userName" column="user_name"/>
</association>
</resultMap>
<select id="queryList" resultMap="appResult">
SELECT
app.*,
cu.id as cu_id,
cu.name as cu_name,
cu.user_name as cu_user_name,
uu.id as uu_id,
uu.name as uu_name,
uu.user_name as uu_user_name
FROM
application app
left join sys_user cu on app.create_user_id = cu.id
left join sys_user uu on app.update_user_id = uu.id
<where>
<if test="params.ids != null and params.ids != ''">
app.id in
<foreach item="id" collection="params.ids.split(',')" separator="," open="(" close=")">
#{id}
</foreach>
</if>
<if test="params.q != null and params.q != ''">
AND ( locate(#{params.q}, app.name) OR locate(#{params.q}, app.description) )
</if>
<if test="params.workspaceId != null and params.workspaceId != ''">
AND app.workspace_id = #{params.workspaceId}
</if>
</where>
<if test="params.orderBy == null or params.orderBy == ''">
ORDER BY app.id
</if>
</select>
</mapper>

View File

@@ -0,0 +1,59 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd">
<mapper namespace="net.geedge.asw.module.app.dao.ApplicationMergeDao">
<!-- 通用查询映射结果 -->
<resultMap id="resultMap" type="net.geedge.asw.module.app.entity.ApplicationMergeEntity">
<id column="id" property="id"/>
<result column="source_branch" property="sourceBranch"/>
<result column="target_branch" property="targetBranch"/>
<result column="start_commit_id" property="startCommitId"/>
<result column="end_commit_id" property="endCommitId"/>
<result column="title" property="title"/>
<result column="remove_source_branch" property="removeSourceBranch"/>
<result column="description" property="description"/>
<result column="status" property="status"/>
<result column="message" property="message"/>
<result column="create_timestamp" property="createTimestamp"/>
<result column="create_user_id" property="createUserId"/>
<result column="workspace_id" property="workspaceId"/>
<association property="createUser" columnPrefix="cu_" javaType="net.geedge.asw.module.sys.entity.SysUserEntity">
<id property="id" column="id"/>
<result property="name" column="name"/>
<result property="userName" column="user_name"/>
</association>
</resultMap>
<select id="queryList" resultMap="resultMap">
SELECT
mr.*,
cu.id AS cu_id,
cu.name AS cu_name,
cu.user_name AS cu_user_name
FROM
application_merge mr
LEFT JOIN sys_user cu ON mr.create_user_id = cu.id
<where>
<if test="params.workspaceId != null and params.workspaceId != ''">
mr.workspace_id = #{params.workspaceId}
</if>
<if test="params.q != null and params.q != ''">
AND ( locate(#{params.q}, mr.title) OR locate(#{params.q}, mr.description) )
</if>
</where>
GROUP BY mr.id
<if test="params.orderBy == null or params.orderBy == ''">
ORDER BY mr.create_timestamp DESC
</if>
</select>
</mapper>

View File

@@ -0,0 +1,55 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd">
<mapper namespace="net.geedge.asw.module.app.dao.ApplicationReleaseDao">
<!-- 通用查询映射结果 -->
<resultMap id="resultMap" type="net.geedge.asw.module.app.entity.ApplicationReleaseEntity">
<id column="id" property="id"/>
<result column="name" property="name"/>
<result column="tag_name" property="tagName"/>
<result column="path" property="path"/>
<result column="description" property="description"/>
<result column="create_timestamp" property="createTimestamp"/>
<result column="create_user_id" property="createUserId"/>
<result column="update_timestamp" property="updateTimestamp"/>
<result column="update_user_id" property="updateUserId"/>
<result column="workspace_id" property="workspaceId"/>
<association property="createUser" columnPrefix="c_" javaType="net.geedge.asw.module.sys.entity.SysUserEntity">
<id property="id" column="id"/>
<result property="name" column="name"/>
<result property="userName" column="user_name"/>
</association>
<association property="updateUser" columnPrefix="u_" javaType="net.geedge.asw.module.sys.entity.SysUserEntity">
<id property="id" column="id"/>
<result property="name" column="name"/>
<result property="userName" column="user_name"/>
</association>
</resultMap>
<select id="queryList" resultMap="resultMap">
SELECT
ar.*,
c.id AS c_id,
c.name AS c_name,
c.user_name AS c_user_name,
u.id AS u_id,
u.name AS u_name,
u.user_name AS u_user_name
FROM
application_release ar
LEFT JOIN sys_user c ON ar.create_user_id = c.id
LEFT JOIN sys_user u ON ar.update_user_id = u.id
<where>
<if test="params.workbookId != null and params.workbookId != ''">
ar.workbook_id = #{params.workbookId}
</if>
<if test="params.q != null and params.q != ''">
AND ( locate(#{params.q}, ar.name) OR locate(#{params.q}, ar.tag_name) OR locate(#{params.q}, ar.description) )
</if>
</where>
</select>
</mapper>

View File

@@ -0,0 +1,142 @@
<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd" >
<mapper namespace="net.geedge.asw.module.job.dao.JobCfgDao">
<resultMap type="net.geedge.asw.module.job.entity.JobCfgEntity" id="jobCfgResultMap">
<id property="id" column="id"/>
<result property="name" column="name"/>
<result property="description" column="description"/>
<result property="packageId" column="package_id"/>
<result property="envId" column="env_id"/>
<result property="playbookId" column="playbook_id"/>
<result property="type" column="type"/>
<result property="cron" column="cron"/>
<result property="status" column="status"/>
<result property="parameters" column="parameters"/>
<result property="createTimestamp" column="create_timestamp"/>
<result property="updateTimestamp" column="update_timestamp"/>
<result property="createUserId" column="create_user_id"/>
<result property="updateUserId" column="update_user_id"/>
<result property="workspaceId" column="workspace_id"/>
<association property="pkg" columnPrefix="pkg_" javaType="net.geedge.asw.module.app.entity.PackageEntity">
<id property="id" column="id"/>
<result property="platform" column="platform"/>
<result property="version" column="version"/>
<result property="name" column="name"/>
</association>
<association property="environment" columnPrefix="em_" javaType="net.geedge.asw.module.environment.entity.EnvironmentEntity">
<id property="id" column="id"/>
<result property="name" column="name"/>
</association>
<association property="playbook" columnPrefix="pb_" javaType="net.geedge.asw.module.job.entity.PlaybookEntity">
<id property="id" column="id"/>
<result property="name" column="name"/>
</association>
<association property="lastJobResult" columnPrefix="job_" javaType="net.geedge.asw.module.job.entity.JobEntity">
<id property="id" column="id"/>
<result property="status" column="status"/>
<result property="createTimestamp" column="create_timestamp"/>
<result property="startTimestamp" column="start_timestamp"/>
<result property="endTimestamp" column="end_timestamp"/>
</association>
<association property="createUser" columnPrefix="cu_" javaType="net.geedge.asw.module.sys.entity.SysUserEntity">
<id property="id" column="id"/>
<result property="name" column="name"/>
</association>
<association property="updateUser" columnPrefix="uu_" javaType="net.geedge.asw.module.sys.entity.SysUserEntity">
<id property="id" column="id"/>
<result property="name" column="name"/>
</association>
<association property="workspace" columnPrefix="ws_" javaType="net.geedge.asw.module.workspace.entity.WorkspaceEntity">
<id property="id" column="id"/>
<result property="name" column="name"/>
</association>
<collection property="pcap" columnPrefix="pcap_" javaType="java.util.List" ofType="net.geedge.asw.module.job.entity.PcapEntity">
<id property="id" column="id"/>
<result property="name" column="name"/>
</collection>
</resultMap>
<select id="queryList" resultMap="jobCfgResultMap">
SELECT
jc.*,
pkg.id AS pkg_id,
pkg.platform AS pkg_platform,
pkg.version AS pkg_version,
pkg.name AS pkg_name,
env.id AS em_id,
env.name AS em_name,
pb.id AS pb_id,
pb.name AS pb_name,
job.id AS job_id,
job.status AS job_status,
job.create_timestamp AS job_create_timestamp,
job.start_timestamp AS job_start_timestamp,
job.end_timestamp AS job_end_timestamp,
pcap.id AS pcap_id,
pcap.name AS pcap_name,
cu.id AS cu_id,
cu.name AS cu_name,
uu.id AS uu_id,
uu.name AS uu_name,
ws.id AS ws_id,
ws.name AS ws_name
FROM
job_cfg jc
LEFT JOIN environment env ON jc.env_id = env.id
LEFT JOIN package pkg ON jc.package_id = pkg.id
LEFT JOIN playbook pb ON jc.playbook_id = pb.id
LEFT JOIN sys_user cu ON jc.create_user_id = cu.id
LEFT JOIN sys_user uu ON jc.update_user_id = uu.id
LEFT JOIN job job ON jc.id = job.job_cfg_id AND job.create_timestamp = (SELECT MAX(create_timestamp) FROM job j WHERE j.job_cfg_id = jc.id)
LEFT JOIN pcap pcap ON job.id = pcap.job_id
LEFT JOIN workspace ws ON jc.workspace_id = ws.id
<where>
<if test="params.ids != null and params.ids != ''">
jc.id in
<foreach item="id" collection="params.ids.split(',')" separator="," open="(" close=")">#{id}</foreach>
</if>
<if test="params.packageIds != null and params.packageIds != ''">
AND pkg.id in
<foreach item="id" collection="params.packageIds.split(',')" separator="," open="(" close=")">#{id}</foreach>
</if>
<if test="params.environmentIds != null and params.environmentIds != ''">
AND env.id in
<foreach item="id" collection="params.environmentIds.split(',')" separator="," open="(" close=")">#{id}</foreach>
</if>
<if test="params.playbooks != null and params.playbooks != ''">
AND pb.id in
<foreach item="id" collection="params.playbooks.split(',')" separator="," open="(" close=")">#{id}</foreach>
</if>
<if test="params.workspaceId != null and params.workspaceId != ''">
AND jc.workspace_id = #{params.workspaceId}
</if>
<if test="params.q != null and params.q != ''">
AND ( locate(#{params.q}, pkg.name) OR locate(#{params.q}, env.name) OR locate(#{params.q}, pb.name) )
</if>
</where>
<if test="params.orderBy == null or params.orderBy == ''">
ORDER BY jc.id
</if>
</select>
</mapper>

View File

@@ -1,18 +1,20 @@
<?xml version="1.0" encoding="UTF-8"?> <?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd"> <!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd">
<mapper namespace="net.geedge.asw.module.runner.dao.JobDao"> <mapper namespace="net.geedge.asw.module.job.dao.JobDao">
<resultMap type="net.geedge.asw.module.runner.entity.JobEntity" id="jobResultMap"> <resultMap type="net.geedge.asw.module.job.entity.JobEntity" id="jobResultMap">
<id property="id" column="id"/> <id property="id" column="id"/>
<result property="jobCfgId" column="job_cfg_id"/>
<result property="playbookId" column="playbook_id"/> <result property="playbookId" column="playbook_id"/>
<result property="packageId" column="package_id"/> <result property="packageId" column="package_id"/>
<result property="envId" column="env_id"/> <result property="envId" column="env_id"/>
<result property="parameters" column="parameters"/>
<result property="startTimestamp" column="start_timestamp"/> <result property="startTimestamp" column="start_timestamp"/>
<result property="endTimestamp" column="end_timestamp"/> <result property="endTimestamp" column="end_timestamp"/>
<result property="status" column="status"/> <result property="status" column="status"/>
<result property="pcapId" column="pcap_id"/>
<result property="logPath" column="log_path"/> <result property="logPath" column="log_path"/>
<result property="artifactsPath" column="artifacts_path"/>
<result property="createTimestamp" column="create_timestamp"/> <result property="createTimestamp" column="create_timestamp"/>
<result property="updateTimestamp" column="update_timestamp"/> <result property="updateTimestamp" column="update_timestamp"/>
<result property="createUserId" column="create_user_id"/> <result property="createUserId" column="create_user_id"/>
@@ -31,7 +33,22 @@
<result property="name" column="name"/> <result property="name" column="name"/>
</association> </association>
<association property="playbook" columnPrefix="pb_" javaType="net.geedge.asw.module.runner.entity.PlaybookEntity"> <association property="playbook" columnPrefix="pb_" javaType="net.geedge.asw.module.job.entity.PlaybookEntity">
<id property="id" column="id"/>
<result property="name" column="name"/>
</association>
<association property="workspace" columnPrefix="ws_" javaType="net.geedge.asw.module.workspace.entity.WorkspaceEntity">
<id property="id" column="id"/>
<result property="name" column="name"/>
</association>
<association property="jobCfg" columnPrefix="jc_" javaType="net.geedge.asw.module.job.entity.JobCfgEntity">
<id property="id" column="id"/>
<result property="name" column="name"/>
</association>
<association property="createUser" columnPrefix="cu_" javaType="net.geedge.asw.module.sys.entity.SysUserEntity">
<id property="id" column="id"/> <id property="id" column="id"/>
<result property="name" column="name"/> <result property="name" column="name"/>
</association> </association>
@@ -50,6 +67,15 @@
env.id AS em_id, env.id AS em_id,
env.name AS em_name, env.name AS em_name,
jc.id AS jc_id,
jc.name AS jc_name,
ws.id AS ws_id,
ws.name AS ws_name,
su.id AS cu_id,
su.name AS cu_name,
pb.id AS pb_id, pb.id AS pb_id,
pb.name AS pb_name pb.name AS pb_name
FROM FROM
@@ -57,6 +83,9 @@
LEFT JOIN environment env ON job.env_id = env.id LEFT JOIN environment env ON job.env_id = env.id
LEFT JOIN package pkg ON job.package_id = pkg.id LEFT JOIN package pkg ON job.package_id = pkg.id
LEFT JOIN playbook pb ON job.playbook_id = pb.id LEFT JOIN playbook pb ON job.playbook_id = pb.id
LEFT JOIN job_cfg jc ON job.job_cfg_id = jc.id
LEFT JOIN workspace ws ON job.workspace_id = ws.id
LEFT JOIN sys_user su ON job.create_user_id = su.id
<where> <where>
<if test="params.ids != null and params.ids != ''"> <if test="params.ids != null and params.ids != ''">
job.id in job.id in
@@ -73,6 +102,11 @@
<foreach item="id" collection="params.environmentIds.split(',')" separator="," open="(" close=")">#{id}</foreach> <foreach item="id" collection="params.environmentIds.split(',')" separator="," open="(" close=")">#{id}</foreach>
</if> </if>
<if test="params.jobCfgIds != null and params.jobCfgIds != ''">
AND jc.id in
<foreach item="id" collection="params.jobCfgIds.split(',')" separator="," open="(" close=")">#{id}</foreach>
</if>
<if test="params.playbooks != null and params.playbooks != ''"> <if test="params.playbooks != null and params.playbooks != ''">
AND pb.id in AND pb.id in
<foreach item="id" collection="params.playbooks.split(',')" separator="," open="(" close=")">#{id}</foreach> <foreach item="id" collection="params.playbooks.split(',')" separator="," open="(" close=")">#{id}</foreach>
@@ -80,6 +114,9 @@
<if test="params.workspaceId != null and params.workspaceId != ''"> <if test="params.workspaceId != null and params.workspaceId != ''">
AND job.workspace_id = #{params.workspaceId} AND job.workspace_id = #{params.workspaceId}
</if> </if>
<if test="params.q != null and params.q != ''">
AND ( locate(#{params.q}, pkg.name) OR locate(#{params.q}, env.name) OR locate(#{params.q}, pb.name) )
</if>
</where> </where>
GROUP BY GROUP BY

View File

@@ -1,9 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?> <?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd"> <!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd">
<mapper namespace="net.geedge.asw.module.runner.dao.PcapDao"> <mapper namespace="net.geedge.asw.module.job.dao.PcapDao">
<resultMap type="net.geedge.asw.module.runner.entity.PcapEntity" id="pcapResultMap"> <resultMap type="net.geedge.asw.module.job.entity.PcapEntity" id="pcapResultMap">
<id property="id" column="id"/> <id property="id" column="id"/>
<result property="name" column="name"/> <result property="name" column="name"/>
<result property="description" column="description"/> <result property="description" column="description"/>
@@ -15,8 +15,7 @@
<result property="createTimestamp" column="create_timestamp"/> <result property="createTimestamp" column="create_timestamp"/>
<result property="createUserId" column="create_user_id"/> <result property="createUserId" column="create_user_id"/>
<result property="workspaceId" column="workspace_id"/> <result property="workspaceId" column="workspace_id"/>
<result property="jobId" column="job_id"/>
<result property="jobId" column="jobId"/>
<association property="pkg" columnPrefix="pkg_" javaType="net.geedge.asw.module.app.entity.PackageEntity"> <association property="pkg" columnPrefix="pkg_" javaType="net.geedge.asw.module.app.entity.PackageEntity">
<id property="id" column="id"/> <id property="id" column="id"/>
@@ -30,11 +29,17 @@
<result property="name" column="name"/> <result property="name" column="name"/>
</association> </association>
<association property="playbook" columnPrefix="pb_" javaType="net.geedge.asw.module.runner.entity.PlaybookEntity"> <association property="playbook" columnPrefix="pb_" javaType="net.geedge.asw.module.job.entity.PlaybookEntity">
<id property="id" column="id"/> <id property="id" column="id"/>
<result property="name" column="name"/> <result property="name" column="name"/>
</association> </association>
<association property="createUser" columnPrefix="cu_" javaType="net.geedge.asw.module.sys.entity.SysUserEntity">
<id property="id" column="id"/>
<result property="name" column="name"/>
<result property="userName" column="user_name"/>
</association>
</resultMap> </resultMap>
<select id="queryList" resultMap="pcapResultMap"> <select id="queryList" resultMap="pcapResultMap">
@@ -50,13 +55,17 @@
env.name AS em_name, env.name AS em_name,
pb.id AS pb_id, pb.id AS pb_id,
pb.name AS pb_name pb.name AS pb_name,
su.id AS cu_id,
su.name AS cu_name,
su.user_name AS cu_user_name
FROM FROM
pcap pcap pcap pcap
left join job job on pcap.id = job.pcap_id left join job job on pcap.job_id = job.id
LEFT JOIN environment env ON job.env_id = env.id LEFT JOIN environment env ON job.env_id = env.id
LEFT JOIN package pkg ON job.package_id = pkg.id LEFT JOIN package pkg ON job.package_id = pkg.id
LEFT JOIN playbook pb ON job.playbook_id = pb.id LEFT JOIN playbook pb ON job.playbook_id = pb.id
LEFT JOIN sys_user su ON su.id = pb.create_user_id
<where> <where>
<if test="params.ids != null and params.ids != ''"> <if test="params.ids != null and params.ids != ''">
pcap.id in pcap.id in

Some files were not shown because too many files have changed in this diff Show More