58 Commits

Author SHA1 Message Date
qidaijie
f80beb8324 修复Json解析为Map时函数处理逻辑。 2021-07-17 16:54:59 +08:00
qidaijie
2ccc0d77a3 修改Storm初始化执行逻辑 2021-07-14 14:13:51 +08:00
qidaijie
3a60e16e71 1:新增日志字段类型弱校验功能,根据Schema定义的字段类型对原始日志不匹配的字段进行类型转换。
2:增加complete.check.type配置对此功能进行开关。
2021-07-14 13:57:40 +08:00
qidaijie
163b724a12 1:新增日志字段类型弱校验功能,根据Schema定义的字段类型对原始日志不匹配的字段进行类型转换。
2:增加complete.check.type配置对此功能进行开关。
2021-07-14 13:56:54 +08:00
qidaijie
5764bb999e 1:修改JSON解析方式为JackSon。
2:删除Kafka生产者flush方法。
3:修改打包方式为package。
2021-07-07 15:19:34 +08:00
qidaijie
1b47ecf76b 1:增加default配置文件。
2:修复解析异常程序终止性bug。
2021-06-11 11:16:26 +08:00
qidaijie
d7f3e40340 1:增加default配置文件。
2:修复解析异常程序终止性bug。
2021-06-11 11:10:16 +08:00
qidaijie
88ffda19a2 1:增加app_match函数。
2:配置文件增加 appId 查询URL。
3:整理配置文件以及readme。
2021-04-23 18:06:37 +08:00
戚岱杰
668ed010af Merge branch 'v3.20.12.12-ack-ratelimit-con-eal4' into 'master'
V3.20.12.12 ack ratelimit con eal4

See merge request bigdata/tsg/log-stream-completion!4
2021-04-23 02:14:42 +00:00
qidaijie
a07c9d0ba7 修改pom版本信息 2021-04-01 15:41:26 +08:00
qidaijie
0023a1455f 修复EAL4中低级警告版本 2021-03-25 14:27:41 +08:00
qidaijie
85f8978887 增加 每次打包的jar包md5值相同 2021-03-22 11:39:55 +08:00
qidaijie
21b150c2bb 修复EAL4中高级警告版本 2021-03-16 14:48:07 +08:00
qidaijie
f0ed75b375 修复EAL4中高级警告版本 2021-03-16 10:39:23 +08:00
qidaijie
51fb19a597 增加 set 函数。 2021-02-01 11:05:02 +08:00
qidaijie
aff082a303 修改if函数,判断是否为数字若为数字则转换为long类型返回结果。 2020-12-27 18:15:06 +08:00
qidaijie
e6c602a154 提交线上使用版本 2020-12-25 17:32:54 +08:00
李玺康
5438144b07 Merge branch 'revert-2c448492' into 'master'
Revert "Merge branch 'tsg_galaxy_com_schema_v3.0.20191217' into 'master'"

See merge request bigdata/tsg/log-stream-completion!3
2020-02-25 15:13:14 +08:00
李玺康
94f14a0c59 Revert "Merge branch 'tsg_galaxy_com_schema_v3.0.20191217' into 'master'"
This reverts merge request !2
2020-02-25 15:12:56 +08:00
李玺康
2c448492c6 Merge branch 'tsg_galaxy_com_schema_v3.0.20191217' into 'master'
Tsg galaxy com schema v3.0.20191217

See merge request bigdata/tsg/log-stream-completion!2
2020-02-25 15:12:23 +08:00
李玺康
31456351a3 Deleted .idea/artifacts/completeUtil.xml, .idea/compiler.xml, .idea/encodings.xml, .idea/vcs.xml, out/artifacts/completeUtil/completeUtil.jar files 2020-02-25 15:12:03 +08:00
lee
4145f2cdfd Merge remote-tracking branch 'origin/tsg_galaxy_com_schema_v3.0.20191217' into tsg_galaxy_com_schema_v3.0.20191217
# Conflicts:
#	src/main/java/cn/ac/iie/utils/general/TransFormUtils.java
#	src/main/java/cn/ac/iie/utils/hbase/HBaseUtils.java
2020-02-25 14:17:03 +08:00
lee
28e0b9e38c 动态获取schema代码更新0225 2020-02-25 14:16:16 +08:00
lee
475f28cabc 工具类打包替换 2020-02-12 16:32:26 +08:00
lee
13cae17743 工具类打成jar包 2020-02-06 14:28:54 +08:00
lee
7e9eb8d4b9 工具类打成jar包 2020-02-06 14:28:17 +08:00
lee
6e452dab29 添加http依赖 2020-02-04 10:57:43 +08:00
李玺康
fcbf3824ba 动态获取schema代码更新 2019-12-20 17:42:51 +08:00
李玺康
a12f82ae8a 动态获取schema代码更新 2019-12-20 12:49:30 +08:00
李玺康
b1cbb95ed1 动态获取schema代码更新 2019-12-19 17:38:20 +08:00
李玺康
3245ac7507 动态获取schema代码更新 2019-12-19 15:45:51 +08:00
李玺康
00b8f6d2d2 动态获取schema代码更新 2019-12-17 14:42:30 +08:00
李玺康
98a4a66a5f 动态获取schema代码更新 2019-12-17 14:31:30 +08:00
qidaijie
a7e25af875 修改 domain获取方式为 使用guava获取 2019-12-17 10:42:06 +08:00
qidaijie
fb22d60bfe 替换某些字段类型为Integer类型,防止int类型为0影响本身语义 2019-12-06 18:50:31 +08:00
qidaijie
7aac9e03d5 公共字段增加 common_sub_action 2019-12-03 14:00:44 +08:00
qidaijie
57fbb78d11 更新 domain 解析正则匹配错误问题 2019-11-28 11:53:35 +08:00
李玺康
2efd619314 Deleted src/main/java/cn/ac/iie/bean/CollectProtocolRecordLog.java, src/main/java/cn/ac/iie/bean/ProxySessionRecordLog.java, src/main/java/cn/ac/iie/bean/RadiusSessionRecordLog.java, src/main/java/cn/ac/iie/bean/SecurityPolicyLog.java files 2019-11-15 15:52:06 +08:00
李玺康
803deb823e Merge remote-tracking branch 'origin/tsg_galaxy_com_v3.0.2019115' into tsg_galaxy_com_v3.0.2019115 2019-11-15 11:43:40 +08:00
李玺康
784a426ee1 流的追踪ID字段类型改变 2019-11-15 11:43:06 +08:00
qidaijie
8f59751da9 更新hbase配置 2019-11-15 09:49:51 +08:00
李玺康
692eabd056 Merge remote-tracking branch 'origin/tsg_galaxy_com_v3.0.2019115' into tsg_galaxy_com_v3.0.2019115 2019-11-13 16:52:17 +08:00
李玺康
97148a3ad0 日志补全代码更新 2019-11-13 16:51:50 +08:00
李玺康
89cf61d0a9 Deleted .idea/libraries/Maven__asm_asm_3_1.xml, .idea/libraries/Maven__com_101tec_zkclient_0_10.xml, .idea/libraries/Maven__com_alibaba_fastjson_1_2_47.xml, .idea/libraries/Maven__com_esotericsoftware_kryo_3_0_3.xml, .idea/libraries/Maven__com_esotericsoftware_minlog_1_3_0.xml, .idea/libraries/Maven__com_esotericsoftware_reflectasm_1_10_1.xml, .idea/libraries/Maven__com_fasterxml_jackson_core_jackson_annotations_2_9_0.xml, .idea/libraries/Maven__com_fasterxml_jackson_core_jackson_core_2_9_1.xml, .idea/libraries/Maven__com_fasterxml_jackson_core_jackson_databind_2_9_1.xml, .idea/libraries/Maven__com_github_stephenc_findbugs_findbugs_annotations_1_3_9_1.xml, .idea/libraries/Maven__com_google_code_findbugs_jsr305_3_0_0.xml, .idea/libraries/Maven__com_google_code_gson_gson_2_2_4.xml, .idea/libraries/Maven__com_google_guava_guava_18_0.xml, .idea/libraries/Maven__com_google_protobuf_protobuf_java_2_5_0.xml, .idea/libraries/Maven__com_googlecode_json_simple_json_simple_1_1.xml, .idea/libraries/Maven__com_jamesmurty_utils_java_xmlbuilder_0_4.xml, .idea/libraries/Maven__com_jcraft_jsch_0_1_42.xml, .idea/libraries/Maven__com_lmax_disruptor_3_3_2.xml, .idea/libraries/Maven__com_maxmind_db_maxmind_db_1_2_2.xml, .idea/libraries/Maven__com_maxmind_geoip2_geoip2_2_12_0.xml, .idea/libraries/Maven__com_maxmind_geoip_geoip_api_1_3_1.xml, .idea/libraries/Maven__com_squareup_okhttp_okhttp_2_4_0.xml, .idea/libraries/Maven__com_squareup_okio_okio_1_4_0.xml, .idea/libraries/Maven__com_squareup_retrofit_retrofit_1_9_0.xml, .idea/libraries/Maven__com_sun_jersey_jersey_core_1_9.xml, .idea/libraries/Maven__com_sun_jersey_jersey_json_1_9.xml, .idea/libraries/Maven__com_sun_jersey_jersey_server_1_9.xml, .idea/libraries/Maven__com_sun_xml_bind_jaxb_impl_2_2_3_1.xml, .idea/libraries/Maven__com_thoughtworks_paranamer_paranamer_2_3.xml, .idea/libraries/Maven__com_yammer_metrics_metrics_core_2_2_0.xml, .idea/libraries/Maven__com_zdjizhi_galaxy_1_0_1.xml, .idea/libraries/Maven__commons_beanutils_commons_beanutils_1_7_0.xml, .idea/libraries/Maven__commons_beanutils_commons_beanutils_core_1_8_0.xml, .idea/libraries/Maven__commons_cli_commons_cli_1_2.xml, .idea/libraries/Maven__commons_codec_commons_codec_1_9.xml, .idea/libraries/Maven__commons_collections_commons_collections_3_2_2.xml, .idea/libraries/Maven__commons_configuration_commons_configuration_1_6.xml, .idea/libraries/Maven__commons_daemon_commons_daemon_1_0_13.xml, .idea/libraries/Maven__commons_digester_commons_digester_1_8.xml, .idea/libraries/Maven__commons_el_commons_el_1_0.xml, .idea/libraries/Maven__commons_httpclient_commons_httpclient_3_1.xml, .idea/libraries/Maven__commons_io_commons_io_2_5.xml, .idea/libraries/Maven__commons_lang_commons_lang_2_5.xml, .idea/libraries/Maven__commons_logging_commons_logging_1_2.xml, .idea/libraries/Maven__commons_net_commons_net_3_1.xml, .idea/libraries/Maven__io_dropwizard_metrics_metrics_core_3_1_2.xml, .idea/libraries/Maven__io_netty_netty_3_10_5_Final.xml, .idea/libraries/Maven__io_netty_netty_all_4_1_8_Final.xml, .idea/libraries/Maven__javax_activation_activation_1_1.xml, .idea/libraries/Maven__javax_servlet_jsp_jsp_api_2_1.xml, .idea/libraries/Maven__javax_servlet_servlet_api_2_5.xml, .idea/libraries/Maven__javax_xml_bind_jaxb_api_2_2_2.xml, .idea/libraries/Maven__javax_xml_stream_stax_api_1_0_2.xml, .idea/libraries/Maven__jline_jline_0_9_94.xml, .idea/libraries/Maven__joda_time_joda_time_2_10.xml, .idea/libraries/Maven__junit_junit_4_12.xml, .idea/libraries/Maven__log4j_log4j_1_2_14.xml, .idea/libraries/Maven__net_java_dev_jets3t_jets3t_0_9_0.xml, .idea/libraries/Maven__net_sf_jopt_simple_jopt_simple_5_0_4.xml, .idea/libraries/Maven__org_apache_avro_avro_1_7_4.xml, .idea/libraries/Maven__org_apache_commons_commons_compress_1_4_1.xml, .idea/libraries/Maven__org_apache_commons_commons_lang3_3_4.xml, .idea/libraries/Maven__org_apache_commons_commons_math3_3_1_1.xml, .idea/libraries/Maven__org_apache_commons_commons_math_2_2.xml, .idea/libraries/Maven__org_apache_curator_curator_client_2_7_1.xml, .idea/libraries/Maven__org_apache_curator_curator_framework_2_10_0.xml, .idea/libraries/Maven__org_apache_curator_curator_recipes_2_7_1.xml, .idea/libraries/Maven__org_apache_directory_api_api_asn1_api_1_0_0_M20.xml, .idea/libraries/Maven__org_apache_directory_api_api_util_1_0_0_M20.xml, .idea/libraries/Maven__org_apache_directory_server_apacheds_i18n_2_0_0_M15.xml, .idea/libraries/Maven__org_apache_directory_server_apacheds_kerberos_codec_2_0_0_M15.xml, .idea/libraries/Maven__org_apache_hadoop_hadoop_annotations_2_7_1.xml, .idea/libraries/Maven__org_apache_hadoop_hadoop_auth_2_7_4.xml, .idea/libraries/Maven__org_apache_hadoop_hadoop_client_2_7_1.xml, .idea/libraries/Maven__org_apache_hadoop_hadoop_common_2_7_1.xml, .idea/libraries/Maven__org_apache_hadoop_hadoop_hdfs_2_7_1.xml, .idea/libraries/Maven__org_apache_hadoop_hadoop_mapreduce_client_app_2_7_1.xml, .idea/libraries/Maven__org_apache_hadoop_hadoop_mapreduce_client_common_2_7_1.xml, .idea/libraries/Maven__org_apache_hadoop_hadoop_mapreduce_client_core_2_7_4.xml, .idea/libraries/Maven__org_apache_hadoop_hadoop_mapreduce_client_jobclient_2_7_1.xml, .idea/libraries/Maven__org_apache_hadoop_hadoop_mapreduce_client_shuffle_2_7_1.xml, .idea/libraries/Maven__org_apache_hadoop_hadoop_yarn_api_2_7_1.xml, .idea/libraries/Maven__org_apache_hadoop_hadoop_yarn_client_2_7_1.xml, .idea/libraries/Maven__org_apache_hadoop_hadoop_yarn_common_2_7_4.xml, .idea/libraries/Maven__org_apache_hadoop_hadoop_yarn_server_common_2_7_1.xml, .idea/libraries/Maven__org_apache_hbase_hbase_annotations_1_4_9.xml, .idea/libraries/Maven__org_apache_hbase_hbase_client_1_4_9.xml, .idea/libraries/Maven__org_apache_hbase_hbase_common_1_4_9.xml, .idea/libraries/Maven__org_apache_hbase_hbase_common_tests_1_4_9.xml, .idea/libraries/Maven__org_apache_hbase_hbase_hadoop2_compat_1_4_9.xml, .idea/libraries/Maven__org_apache_hbase_hbase_hadoop_compat_1_4_9.xml, .idea/libraries/Maven__org_apache_hbase_hbase_metrics_1_4_9.xml, .idea/libraries/Maven__org_apache_hbase_hbase_metrics_api_1_4_9.xml, .idea/libraries/Maven__org_apache_hbase_hbase_prefix_tree_1_4_9.xml, .idea/libraries/Maven__org_apache_hbase_hbase_procedure_1_4_9.xml, .idea/libraries/Maven__org_apache_hbase_hbase_protocol_1_4_9.xml, .idea/libraries/Maven__org_apache_hbase_hbase_server_1_4_9.xml, .idea/libraries/Maven__org_apache_htrace_htrace_core_3_1_0_incubating.xml, .idea/libraries/Maven__org_apache_httpcomponents_httpclient_4_5_2.xml, .idea/libraries/Maven__org_apache_httpcomponents_httpcore_4_4_4.xml, .idea/libraries/Maven__org_apache_kafka_kafka_2_11_1_0_0.xml, .idea/libraries/Maven__org_apache_kafka_kafka_clients_1_0_0.xml, .idea/libraries/Maven__org_apache_logging_log4j_log4j_api_2_1.xml, .idea/libraries/Maven__org_apache_logging_log4j_log4j_core_2_1.xml, .idea/libraries/Maven__org_apache_logging_log4j_log4j_slf4j_impl_2_1.xml, .idea/libraries/Maven__org_apache_storm_storm_core_1_0_2.xml, .idea/libraries/Maven__org_apache_storm_storm_kafka_1_0_2.xml, .idea/libraries/Maven__org_apache_zookeeper_zookeeper_3_4_9.xml, .idea/libraries/Maven__org_clojure_clojure_1_7_0.xml, .idea/libraries/Maven__org_codehaus_jackson_jackson_core_asl_1_9_13.xml, .idea/libraries/Maven__org_codehaus_jackson_jackson_jaxrs_1_9_13.xml, .idea/libraries/Maven__org_codehaus_jackson_jackson_mapper_asl_1_9_13.xml, .idea/libraries/Maven__org_codehaus_jackson_jackson_xc_1_8_3.xml, .idea/libraries/Maven__org_codehaus_jettison_jettison_1_1.xml, .idea/libraries/Maven__org_fusesource_leveldbjni_leveldbjni_all_1_8.xml, .idea/libraries/Maven__org_hamcrest_hamcrest_core_1_3.xml, .idea/libraries/Maven__org_influxdb_influxdb_java_2_1.xml, .idea/libraries/Maven__org_jamon_jamon_runtime_2_4_1.xml, .idea/libraries/Maven__org_jruby_jcodings_jcodings_1_0_8.xml, .idea/libraries/Maven__org_jruby_joni_joni_2_1_2.xml, .idea/libraries/Maven__org_lz4_lz4_java_1_4.xml, .idea/libraries/Maven__org_mortbay_jetty_jetty_6_1_26.xml, .idea/libraries/Maven__org_mortbay_jetty_jetty_sslengine_6_1_26.xml, .idea/libraries/Maven__org_mortbay_jetty_jetty_util_6_1_26.xml, .idea/libraries/Maven__org_mortbay_jetty_jsp_2_1_6_1_14.xml, .idea/libraries/Maven__org_mortbay_jetty_jsp_api_2_1_6_1_14.xml, .idea/libraries/Maven__org_mortbay_jetty_servlet_api_2_5_6_1_14.xml, .idea/libraries/Maven__org_objenesis_objenesis_2_1.xml, .idea/libraries/Maven__org_ow2_asm_asm_5_0_3.xml, .idea/libraries/Maven__org_scala_lang_scala_library_2_11_11.xml, .idea/libraries/Maven__org_slf4j_slf4j_api_1_7_7.xml, .idea/libraries/Maven__org_tukaani_xz_1_0.xml, .idea/libraries/Maven__org_xerial_snappy_snappy_java_1_1_4.xml, .idea/libraries/Maven__tomcat_jasper_compiler_5_5_23.xml, .idea/libraries/Maven__tomcat_jasper_runtime_5_5_23.xml, .idea/libraries/Maven__xerces_xercesImpl_2_9_1.xml, .idea/libraries/Maven__xml_apis_xml_apis_1_3_04.xml, .idea/libraries/Maven__xmlenc_xmlenc_0_52.xml, .idea/libraries, .idea/compiler.xml, .idea/encodings.xml, .idea/log-stream-completion.iml, .idea/misc.xml, .idea/modules.xml, .idea/uiDesigner.xml, .idea/vcs.xml, .idea files 2019-11-12 17:42:11 +08:00
李玺康
0464e9cde5 日志补全代码更新 2019-11-12 17:39:20 +08:00
李玺康
fab47b55fe Merge branch 'tsg_galaxy_com_v3.0.2019115' of https://git.mesalab.cn/bigdata/tsg/log-stream-completion into tsg_galaxy_com_v3.0.2019115 2019-11-12 16:21:44 +08:00
李玺康
56af3618cd radius日志补全代码更新 2019-11-12 16:21:37 +08:00
qidaijie
d83f648292 更新配置文件 2019-11-12 16:02:50 +08:00
李玺康
360c06fe86 V3日志补全代码更新 2019-11-12 14:36:00 +08:00
李玺康
11a389c14f radius日志补全代码更新 2019-11-12 11:29:19 +08:00