gfwleak/galaxy-deployment-updata-record
Archived
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update README.md

Browse Source
This commit is contained in:
窦凤虎
2024-06-18 12:04:00 +00:00
parent ed71c409a5
commit 61c06b7517
1 changed files with 23 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files

40
tsg_olap/installation/flink/groot_stream/README.md
View File

@@ -1,13 +1,26 @@
## session_record.yaml.template
- etl_session_record_kafka_to_ndc_kafka (A-DT) // 多数中心部署：分中心Data Transporter 预处理后，集中汇聚至国家中心（NDC）
- Topology: kafka_source -> etl_processor -> kafka_sink
- Data Flow: SESSION-RECORD -> SESSION-RECORD-PROCESSED
- session_record_processed_kafka_to_clickhouse(A-NDC) // 多数中心部署：国家中心侧加载会话日志写入ClickHouse
- Topology: kafka_source -> clickhouse_sink
- Data Flow: SESSION-RECORD-PROCESSED -> session_record_local
- etl_session_record_kafka_to_clickhouse (B) // 集中部署： 摄入会话日志，预处理后写入ClickHouse
- Topology: kafka_source -> etl_processor -> clickhouse_sink
- Data Flow: SESSION-RECORD -> session_record_local
# 配置模版举例
## session_record.yaml.j2
- 多数中心部署场景: 分中心Data Transporter预处理后，集中汇聚至国家中心（NDC）
- etl_session_record_kafka_to_ndc_kafka (A-DT)
- Topology: kafka_source -> etl_processor -> kafka_sink
- Data Flow: SESSION-RECORD -> SESSION-RECORD-PROCESSED
- 多数中心部署场景：国家中心侧加载会话日志写入ClickHouse
- session_record_processed_kafka_to_clickhouse(A-NDC)
- Topology: kafka_source -> clickhouse_sink
- Data Flow: SESSION-RECORD-PROCESSED -> session_record_local
- 集中部署场景：摄入会话日志，预处理后写入ClickHouse
- etl_session_record_kafka_to_clickhouse (B)
- Topology: kafka_source -> etl_processor -> clickhouse_sink
- Data Flow: SESSION-RECORD -> session_record_local
## datapath_telemetry_record.yaml.j2
- troubleshooting_file_stream_kafka_to_ndc_kafka
- Topology: kafka_source -> kafka_sink (format:raw)
- Data Flow: TROUBLESHOOTING-FILE-STREAM-RECORD -> TROUBLESHOOTING-FILE-STREAM-RECORD
## realtime_log_streaming_cn_session_record.yaml.template
@@ -16,11 +29,4 @@
- etl_session_record_kafka_to_cn_kafka
- Topology: kafka_source -> etl_processor -> post_output_field_processor -> kafka_sink
- Data Flow: SESSION-RECORD(SESSION-RECORD-PROCESSED) -> SESSION-RECORD-CN
## data_transporter.yaml.template
- troubleshooting_file_stream_kafka_to_ndc_kafka
- Topology: kafka_source -> kafka_sink (format:raw)
- Data Flow: TROUBLESHOOTING-FILE-STREAM-RECORD -> TROUBLESHOOTING-FILE-STREAM-RECORD