gfwleak/dongxiaoyan-tsg-autotest
Archived
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

导入导出功能自动化，对象txt格式导入导出，策略txt和json格式导入，txt格式导出

Browse Source
This commit is contained in:
hebingning
2020-12-24 20:02:22 +08:00
parent 2443be3579
commit b11b2a654d
4 changed files with 836 additions and 86 deletions
Download Patch File Download Diff File Expand all files Collapse all files

178
02-Keyword/tsg_bfapi/policy_object/Object.robot
View File

@@ -6,118 +6,117 @@ Resource ../../../02-Keyword/tsg_bfapi/policy_file_interface/Functional
Resource ProcessObjectBody.robot
Library Collections
Resource ../Common.robot
Library ../../../04-CustomLibrary/Library/fileOperations.py
Library DatabaseLibrary
*** Variables ***
${objectUrl} /policy/object
${objectUrl} /policy/object
*** Keywords ***
AddObjects
[Documentation]
... 新增策略对象
[Arguments] ${returnData} ${objectList}
[Documentation] 新增策略对象
... objectList,策略对象,可为list类型与dict类型
... addItemList自动过滤itemId
... updateItemList自动过滤isInitialize(update时该字段引发异常)
... ipItem格式为dict,自动过滤空的字段
... stringItem格式为dict,需要注意keywordArray字段应传入逗号分隔的字符串eg: keyword1,keyword2
... stringItem格式为dict,需要注意keywordArray字段应传入逗号分隔的字符串eg: keyword1,keyword2
... http_signature为代表的拓展关键字keywordArray字段也是以逗号分隔的
... returnData，是否返回数据，固定为1
[Arguments] ${returnData} ${objectList}
${returnData}= Run Keyword If '${returnData}' == '${EMPTY}' Set Variable 1
... ELSE Set Variable ${returnData}
#必选参数判定
Should Not Be Empty ${objectList}
${dictType} = Evaluate type(${objectList})
${body} Run Keyword If "${dictType}" == "<class 'list'>" ObjectListOperation ${returnData} ${objectList} add
... ELSE IF "${dictType}" == "<class 'dict'>" ObjectOperation ${returnData} ${objectList} add
... ELSE IF "${dictType}" == "<class 'dict'>" ObjectOperation ${returnData} ${objectList} add
... ELSE Set Variable ${EMPTY}
${response} BasePostRequestForV2 ${objectUrl} ${body} ${version}
${objectIds} Run Keyword If "${returnData}" == "1" Get-ObjectIds ${response}
... ELSE Create List
${rescode} Set Variable ${response['code']}
${objectIds} Run Keyword If "${returnData}" == "1" Get-ObjectIds ${response}
... ELSE Create List
${rescode} Set Variable ${response['code']}
Should Be Equal As Strings ${rescode} 200
[Return] ${rescode} ${objectIds}
UpdateObjects
[Documentation]
... 更新策略对象
[Arguments] ${returnData} ${objectList} ${opAction}
[Documentation] 更新策略对象
... objectList,策略对象,可为list类型与dict类型
... addItemList自动过滤itemId
... updateItemList自动过滤isInitialize(update时该字段引发异常)
... ipItem格式为dict,自动过滤空的字段
... stringItem格式为dict,需要注意keywordArray字段应传入逗号分隔的字符串eg: keyword1,keyword2
... stringItem格式为dict,需要注意keywordArray字段应传入逗号分隔的字符串eg: keyword1,keyword2
... http_signature为代表的拓展关键字keywordArray字段也是以逗号分隔的
... returnData，是否返回数据，固定为1
... opAction，可为update，enable,disable
[Arguments] ${returnData} ${objectList} ${opAction}
${returnData}= Run Keyword If '${returnData}' == '${EMPTY}' Set Variable 1
... ELSE Set Variable ${returnData}
#必选参数判定
Should Not Be Empty ${objectList}
${dictType} = Evaluate type(${objectList})
${body} Run Keyword If "${dictType}" == "<class 'list'>" ObjectListOperation ${returnData} ${objectList} ${opAction}
... ELSE IF "${dictType}" == "<class 'dict'>" ObjectOperation ${returnData} ${objectList} ${opAction}
... ELSE IF "${dictType}" == "<class 'dict'>" ObjectOperation ${returnData} ${objectList} ${opAction}
... ELSE Set Variable ${EMPTY}
${response} BaseEditRequestForV2 ${objectUrl} ${body} ${version}
${rescode} Set Variable ${response['code']}
${rescode} Set Variable ${response['code']}
Should Be Equal As Strings ${rescode} 200
[Return] ${rescode}
GetObjectItems
[Documentation]
... 获取策略对象中的item,策略对象单元
[Arguments] ${params}
[Documentation] 获取策略对象中的item,策略对象单元
... 当updateItemList中有需要传入数据时
... 调用此关键字
[Arguments] ${params}
#必选参数判定
Should Not Be Empty ${params}
Should Not Be Empty ${params}
${paramsStr} Dictionary-To-QueryParams ${params}
${response} BaseFormRequest /policy/items ${paramsStr} ${version}
${rescode} Set Variable ${response['code']}
${rescode} Set Variable ${response['code']}
Should Be Equal As Strings ${rescode} 200
${data} Set Variable ${response['data']}
${data} Set Variable ${response['data']}
${itemIds} Create List
FOR ${item} IN @{data['list']}
Append To List ${itemIds} ${item['itemId']}
END
[Return] ${rescode} ${itemIds} ${data['list']}
GetObjectItems1
[Documentation]
... 获取策略对象中的item,策略对象单元
[Arguments] ${params}
[Documentation] 获取策略对象中的item,策略对象单元
... 当updateItemList中有需要传入数据时
... 调用此关键字
[Arguments] ${params}
#必选参数判定
Should Not Be Empty ${params}
Should Not Be Empty ${params}
${paramsStr} Dictionary-To-QueryParams ${params}
${response} BaseFormRequest /policy/items ${paramsStr} ${version}
${rescode} Set Variable ${response['code']}
${rescode} Set Variable ${response['code']}
Should Be Equal As Strings ${rescode} 200
${data} Set Variable ${response['data']}
${data} Set Variable ${response['data']}
[Return] ${rescode} ${data}
DeleteObjects
[Documentation]
... 删除策略对象
[Arguments] ${objectIds}
[Documentation] 删除策略对象
... params，传入的删除字典
... 结构为[1,2,3]
[Arguments] ${objectIds}
${type} Evaluate isinstance(${objectIds},list)
Should Be True ${type}
Should Be True ${type}
${dict} Create Dictionary objectIds=${objectIds}
${json} json.Dumps ${dict} ensure_ascii=False
${response} BaseDeleteRequest /${version}${objectUrl} ${json}
${rescode} Set Variable ${response['code']}
Should Be Equal As Strings ${rescode} 200
${rescode} Set Variable ${response['code']}
Should Be Equal As Strings ${rescode} 200
[Return] ${rescode}
ImportObject
[Documentation]
... 策略对象导入
[Arguments] ${filePath} ${fileName} ${objectType} ${objectSubType} ${objectName}
[Documentation] 策略对象导入
... filePath文件路径
... fileName文件名称
... objectType对象类型
... objectSubType对象子类型
... objectName对象名称
... jira CHON-12导入限制需求支持
[Arguments] ${filePath} ${fileName} ${objectType} ${objectSubType} ${objectName}
Directory Should Exist ${path}
File Should Exist ${path}/${filePath}/${fileName}
File Should Not Be Empty ${path}/${filePath}/${fileName}
@@ -126,12 +125,21 @@ ImportObject
#获取Item 的总数
${total} GetItemNum ${objectType} ${objectSubType}
#获取导入文件中的总行数
#${lines} CountLines ${path}/${filePath}/${fileName}
${lines} CountLines ${path}/${filePath}/${fileName}
#总数超过一定数据量测试不通过
${total1} Evaluate ${total}+${lines}
${total1} Evaluate ${total}+${lines}
#链接数据库查询上限
Connect To Database Using Custom Params pymysql ${mysqlHost}
${objectLimit} query SELECT cfg_value FROM t_sys_cfg WHERE cfg_name='${objectType}_upper_limit'
${objectLimit} Set Variable ${objectLimit[0][0]}
${remaining} Evaluate ${objectLimit}-${total}
log ${objectType}剩余可导入量为：${remaining}
${testType} Evaluate type($objectLimit)
Disconnect From Database
#jira CHON-12导入限制需求支持
Run Keyword If "${objectType}"=="url" and ${total1}>1000000 Fail 当前已经存在item个数${total},待导入个数${lines},总数超过100万条！
... ELSE IF ${total1}>100000 Fail 当前已经存在item个数${total},待导入个数${lines},总数超过10万条！
Run Keyword If "${objectType}"=="url" and ${total1}>${objectLimit} log 当前已经存在item个数${total},待导入个数${lines},总数超过${objectLimit}条！
... ELSE IF ${total1}>${objectLimit} log 当前已经存在item个数${total},待导入个数${lines},总数超过${objectLimit}条！
${objectSubType} Run Keyword If "${objectType}"=="ip" and "${objectSubType}"=="${EMPTY}" Set Variable endpoint
... ELSE Set Variable ${objectSubType}
${subfix} Fetch From Right ${fileName} .
@@ -139,33 +147,32 @@ ImportObject
... ELSE IF "${subfix}"=="csv" Evaluate (r'${fileName}',open(r"${path}/${filePath}/${fileName}",'rb'),'application/vnd.ms-excel')
... ELSE ${EMPTY}
${data} Create Dictionary objectType=${objectType}
Run Keyword If "${objectSubType}"!="${EMPTY}" Set To Dictionary ${data} objectSubType=${objectSubType}
Run Keyword If "${objectName}"!="${EMPTY}" Set To Dictionary ${data} objectName=${objectName}
Run Keyword If "${objectSubType}"!="${EMPTY}" Set To Dictionary ${data} objectSubType=${objectSubType}
Run Keyword If "${objectName}"!="${EMPTY}" Set To Dictionary ${data} objectName=${objectName}
${file} Create Dictionary file=${binFile}
${response} BaseMultipartPostRequest ${objectUrl}/batch ${data} ${file} ${version}
${rescode} Set Variable ${response['code']}
Should Be Equal As Strings ${rescode} 200
[Return] ${rescode}
${rescode} Set Variable ${response['code']}
Should Be Equal As Strings ${rescode} 200
[Return] ${rescode} ${objectType} ${response}
ExportObject
[Documentation]
... 策略对象导出
... objectType ip,url等
... objectSubType 子类型
... objectIds 1,2,3
... objectName 字符串
[Arguments] ${objectType} ${objectSubType} ${objectIds} ${objectName}
[Documentation] 策略对象导出
... objectType ip,url等
... objectSubType 子类型
... objectIds 1,2,3
... objectName 字符串
Should Not Be Empty ${objectType}
${params} Create Dictionary objectType=${objectType}
Run Keyword If "${objectSubType}"!="${EMPTY}" Set To Dictionary ${params} objectSubType=${objectSubType}
Run Keyword If "${objectIds}"!="${EMPTY}" Set To Dictionary ${params} objectIds=${objectIds}
Run Keyword If "${objectName}"!="${EMPTY}" Set To Dictionary ${params} objectName=${objectName}
${response} BaseGetRequestReturnBinary ${objectUrl}/batch ${params} ${version}
Run Keyword If "${objectName}"!="${EMPTY}" Set To Dictionary ${params} objectName=${objectName}
${response} BaseGetRequestReturnBinary ${objectUrl}/batch ${params} ${version}
[Return] ${response}
GetItemNum
[Documentation]
... 获取策略对象单元总数
...
[Arguments] ${objectType} ${objectSubType}
[Documentation] 获取策略对象单元总数
${getTotal} Create Dictionary pageNo=${1} pageSize=${1}
Run Keyword If "${objectType}"=="ip" and "${objectSubType}"=="endpoint" Set To Dictionary ${getTotal} itemType=IP
... ELSE IF "${objectType}"=="ip" and "${objectSubType}"=="geo_location" Set To Dictionary ${getTotal} itemType=geo_location
@@ -180,49 +187,56 @@ GetItemNum
${total} Run Keyword If "${objectType}"=="fqdn_category" Evaluate ${total} - ${data['total']}
... ELSE Set Variable ${total}
[Return] ${total}
CountLines
[Documentation]
... 获取文件总行数
CountLines1
[Arguments] ${file}
[Documentation] 获取文件总行数
${count} Set Variable ${0}
${openFile} Evaluate enumerate(open(r"${file}",'r'))
FOR ${line} IN @{openFile}
${val} Run Keyword If "${line[0]}"=="0" Set Variable ${line[1]} ELSE Set Variable ${EMPTY}
${val} Run Keyword If "${line[0]}"=="0" Set Variable ${line[1]}
... ELSE Set Variable ${EMPTY}
${val2} Run Keyword If "${line[1]}"=="\n" Set Variable ${line[0]}
... ELSE Set Variable ${EMPTY}
log ${line}
log "${line[1]}"
${match} Get Regexp Matches ${val} ^-->
#${match2} Get Regexp Matches ${val} ^(\t)*$\n
${len} Get Length ${match}
#${len2} Get Length ${match2}
${count} Run Keyword If "${line[0]}"=="0" and ${len}>0 Set Variable ${count}
... ELSE Evaluate ${count}+1
END
[Return] ${count}
PolicProtocolFields
[Documentation]
... 策略对象生效协议字段查询
[Arguments] ${policyType} ${protocol} ${objectType}
[Documentation] 策略对象生效协议字段查询
... policyType策略类型
... protocol协议
... objectType对象类型
[Arguments] ${policyType} ${protocol} ${objectType}
${dict} Create Dictionary
Run Keyword If "${policyType}"!="${EMPTY}" Set To Dictionary ${dict} policyType=${policyType}
Run Keyword If "${protocol}"!="${EMPTY}" Set To Dictionary ${dict} protocol=${protocol}
Run Keyword If "${objectType}"!="${EMPTY}" Set To Dictionary ${dict} objectType=${objectType}
Run Keyword If "${policyType}"!="${EMPTY}" Set To Dictionary ${dict} policyType=${policyType}
Run Keyword If "${protocol}"!="${EMPTY}" Set To Dictionary ${dict} protocol=${protocol}
Run Keyword If "${objectType}"!="${EMPTY}" Set To Dictionary ${dict} objectType=${objectType}
${parmStr} Dictionary-To-QueryParams ${dict}
${response} BaseFormRequest ${objectUrl}/protocolFields ${parmStr} ${version}
${rescode} Set Variable ${response['code']}
Should Be Equal As Strings ${rescode} 200
${response} BaseFormRequest ${objectUrl}/protocolFields ${parmStr} ${version}
${rescode} Set Variable ${response['code']}
Should Be Equal As Strings ${rescode} 200
[Return] ${response['data']}
ObjectReference
[Documentation]
... 策略对象相关性查询
[Arguments] ${objectId}
[Arguments] ${objectId}
[Documentation] 策略对象相关性查询
${response} BaseFormRequest ${objectUrl}/reference objectId=${objectId} ${version}
${rescode} Set Variable ${response['code']}
Should Be Equal As Strings ${rescode} 200
${rescode} Set Variable ${response['code']}
Should Be Equal As Strings ${rescode} 200
[Return] ${response['data']}
AppIdTreeSearch
[Documentation]
... App ID对象树查询
[Arguments] ${referenceObjectId}
[Documentation] App ID对象树查询
${response} BaseFormRequest ${objectUrl}/appId/tree referenceObjectId=${referenceObjectId} ${version}
${rescode} Set Variable ${response['code']}
Should Be Equal As Strings ${rescode} 200
[Return] ${response['data']}
${rescode} Set Variable ${response['code']}
Should Be Equal As Strings ${rescode} 200
[Return] ${response['data']}

15
02-Keyword/tsg_bfapi/policy_object/Policy.robot
View File

@@ -119,18 +119,25 @@ ImportPolicy
#待导入的策略总数
${totalNum} Get From Dictionary ${fileJson} totalNum
#查询已经导入的非内置策略的数量
${totalExists} GetPolicyTotal 2
${totalExists} GetPolicyTotal 10
${calTotal} Evaluate ${totalNum} + ${totalExists}
Connect To Database Using Custom Params pymysql ${mysqlHost}
${policyLimit} query SELECT cfg_value FROM t_sys_cfg WHERE cfg_name='policy_upper_limit'
${policyLimit} Set Variable ${policyLimit[0][0]}
${remaining} Evaluate ${policyLimit}-${totalExists}
log 策略剩余可导入量为：${remaining}
${testType} Evaluate type($policyLimit)
Disconnect From Database
#jira CHON-12导入限制需求支持
Run Keyword If ${calTotal}>1000000 Fail 当前已经存在策略条数${totalExists},待导入条数${totalNum},总数超过100万条！
Run Keyword If ${calTotal}>${policyLimit} log 当前已经存在策略条数${totalExists},待导入条数${totalNum},总数超过${policyLimit}条！
${binFile} Evaluate (r'import_policy.json',open(r"${path}/${filePath}/${fileName}",'rb'),'application/json')
Log ${binFile}
${data} Create Dictionary policyType=${policyType}
${file} Create Dictionary file=${binFile}
${response} BaseMultipartPostRequest ${policyUrl}/batch ${data} ${file} ${version}
${response} BaseMultipartPostRequest ${policyUrl}/batch ${data} ${file} ${version2}
${rescode} Set Variable ${response['code']}
Should Be Equal As Strings ${rescode} 200
[Return] ${rescode}
[Return] ${rescode} ${totalNum}
ExportPolicy
[Documentation]
... 策略导出