163 lines
7.7 KiB
Markdown
163 lines
7.7 KiB
Markdown
# 亦庄POC环境：百分点希望将日志中的时间精度调整为毫秒格式
|
||
|
||
| ID | Creation Date | Assignee | Status |
|
||
|----|----------------|----------|--------|
|
||
| OMPUB-999 | 2023-08-11T10:33:44.000+0800 | 窦凤虎 | 完成 |
|
||
|
||
|
||
---
|
||
|
||
亦庄P-19 POC环境与百分点调试过程中，他们希望将日志中的时间精度调整为毫秒格式**doufenghu** commented on *2023-08-16T15:27:47.349+0800*:
|
||
|
||
统一时间精度为毫秒方案：
|
||
* 建议方案，增加一列“_ms”，存储毫秒数据。
|
||
** 优点：字段类型无需改动(int64)，计算更高效。现场环境，TTL留存周期过后，可启用毫秒精度字段。
|
||
** 缺点：冗余存储，扩展性不够好。
|
||
* 其它方案，{*}非主键列{*}由int64，调整为浮点数float64
|
||
** 优点：直接兼容秒与毫秒，可扩展性好。
|
||
** 缺点：查询性能和数据压缩比不高，时间函数转换较复杂；在原有环境（E21）修改数据类型效率低，IO会在数天占用100%。
|
||
|
||
其它事项：
|
||
* 查询接口，需要更改为毫秒时间函数。
|
||
* Receive Time 为数据库主键，需要特殊考虑，无法直接更改。
|
||
|
||
|
||
|
||
---
|
||
|
||
**doufenghu** commented on *2023-08-29T11:23:09.489+0800*:
|
||
|
||
结论：原有时间字段，新增一列X_timestamp_ms, 数据类型为Datatime64(3) 存储毫秒级Unix Timestamp。例如：
|
||
* 命名：common_recv_timestamp_ms , common_start_timestamp_ms ....
|
||
* Unix Timestamp（基于函数转换）：1693278926187
|
||
* 字符串日期格式（基于函数转换）：2019-09-16 19:20:12.345
|
||
|
||
|
||
|
||
---
|
||
|
||
**doufenghu** commented on *2023-09-11T17:50:12.263+0800*:
|
||
|
||
Firewall 支持增加毫秒级别会话时间戳 common_start_timestamp_ms， common_end_timestamp_ms 。[~yangwei]
|
||
|
||
|
||
|
||
---
|
||
|
||
**doufenghu** commented on *2023-09-25T11:02:33.547+0800*:
|
||
|
||
因P19 NFS PoC，需要在23.07 增加该功能，TSG在 23.10 展示。
|
||
|
||
|
||
|
||
---
|
||
|
||
**gitlab** commented on *2023-09-27T09:43:14.081+0800*:
|
||
|
||
[刘学利|https://git.mesalab.cn/liuxueli] mentioned this issue in [a commit|https://git.mesalab.cn/tango/tsg_master/-/commit/19656a626a600b5b265eb6b21f5aab5e55f5c537] of [TSG Appliance / tsg_master|https://git.mesalab.cn/tango/tsg_master] on branch [feature-add-timestamp-ms-OMPUB-999|https://git.mesalab.cn/tango/tsg_master/-/tree/feature-add-timestamp-ms-OMPUB-999]:{quote}OMPUB-999: 功能端输出common_start_timestamp_ms，common_end_timestamp_ms日志字段{quote}
|
||
|
||
|
||
|
||
---
|
||
|
||
**gitlab** commented on *2023-09-27T19:16:11.722+0800*:
|
||
|
||
[刘学利|https://git.mesalab.cn/liuxueli] mentioned this issue in [a merge request|https://git.mesalab.cn/tango/tsg_master/-/merge_requests/409] of [TSG Appliance / tsg_master|https://git.mesalab.cn/tango/tsg_master] on branch [feature-add-timestamp-ms-OMPUB-999|https://git.mesalab.cn/tango/tsg_master/-/tree/feature-add-timestamp-ms-OMPUB-999]:{quote}Resolve OMPUB-999 "Feature add timestamp ms "{quote}
|
||
|
||
|
||
|
||
---
|
||
|
||
**gitlab** commented on *2023-09-27T20:09:09.223+0800*:
|
||
|
||
[刘学利|https://git.mesalab.cn/liuxueli] mentioned this issue in [a commit|https://git.mesalab.cn/tsg/tsg-os-buildimage/-/commit/591aed729e55486e75edba2792457b8be9f96da5] of [TSG / tsg-os-buildimage|https://git.mesalab.cn/tsg/tsg-os-buildimage] on branch [update-v23.07-firewall-v11|https://git.mesalab.cn/tsg/tsg-os-buildimage/-/tree/update-v23.07-firewall-v11]:{quote}更新sapp、tsg_master，支持:{quote}
|
||
|
||
|
||
|
||
---
|
||
|
||
**gitlab** commented on *2023-09-27T20:09:44.654+0800*:
|
||
|
||
[刘学利|https://git.mesalab.cn/liuxueli] mentioned this issue in [a merge request|https://git.mesalab.cn/tsg/tsg-os-buildimage/-/merge_requests/1786] of [TSG / tsg-os-buildimage|https://git.mesalab.cn/tsg/tsg-os-buildimage] on branch [update-v23.07-firewall-v11|https://git.mesalab.cn/tsg/tsg-os-buildimage/-/tree/update-v23.07-firewall-v11]:{quote}更新sapp、tsg_master，支持:{quote}
|
||
|
||
|
||
|
||
---
|
||
|
||
**gitlab** commented on *2023-10-11T18:23:05.896+0800*:
|
||
|
||
[刘学利|https://git.mesalab.cn/liuxueli] mentioned this issue in [a commit|https://git.mesalab.cn/tsg/tsg-os-buildimage/-/commit/bda5dcebffc9455a152d023882a58321fbe8695a] of [TSG / tsg-os-buildimage|https://git.mesalab.cn/tsg/tsg-os-buildimage] on branch [update-v23.07-tsg_master|https://git.mesalab.cn/tsg/tsg-os-buildimage/-/tree/update-v23.07-tsg_master]:{quote}更新tsg_master, 支持:{quote}
|
||
|
||
|
||
|
||
---
|
||
|
||
**gitlab** commented on *2023-10-11T18:23:32.057+0800*:
|
||
|
||
[刘学利|https://git.mesalab.cn/liuxueli] mentioned this issue in [a merge request|https://git.mesalab.cn/tsg/tsg-os-buildimage/-/merge_requests/1817] of [TSG / tsg-os-buildimage|https://git.mesalab.cn/tsg/tsg-os-buildimage] on branch [update-v23.07-tsg_master|https://git.mesalab.cn/tsg/tsg-os-buildimage/-/tree/update-v23.07-tsg_master]:{quote}更新tsg_master, 支持:{quote}
|
||
|
||
|
||
|
||
---
|
||
|
||
**gitlab** commented on *2023-10-12T09:59:35.046+0800*:
|
||
|
||
[付明卫|https://git.mesalab.cn/fumingwei] mentioned this issue in [a commit|https://git.mesalab.cn/tsg/tsg-os-buildimage/-/commit/5850781fb197199446a53f08fda9563d4093e465] of [TSG / tsg-os-buildimage|https://git.mesalab.cn/tsg/tsg-os-buildimage] on branch [update-v23.07-tsg_master|https://git.mesalab.cn/tsg/tsg-os-buildimage/-/tree/update-v23.07-tsg_master]:{quote}更新tsg_master, 支持:{quote}
|
||
|
||
|
||
|
||
---
|
||
|
||
**gitlab** commented on *2023-10-13T12:58:16.909+0800*:
|
||
|
||
[刘学利|https://git.mesalab.cn/liuxueli] mentioned this issue in [a commit|https://git.mesalab.cn/tango/tsg_master/-/commit/34834d1a67be40f963ea7cc8881d97a13beb4fe3] of [TSG Appliance / tsg_master|https://git.mesalab.cn/tango/tsg_master] on branch [feature-add-timestamp-ms-OMPUB-999-2|https://git.mesalab.cn/tango/tsg_master/-/tree/feature-add-timestamp-ms-OMPUB-999-2]:{quote}OMPUB-999: 功能端输出common_start_timestamp_ms，common_end_timestamp_ms日志字段{quote}
|
||
|
||
|
||
|
||
---
|
||
|
||
**gitlab** commented on *2023-10-13T13:04:07.643+0800*:
|
||
|
||
[刘学利|https://git.mesalab.cn/liuxueli] mentioned this issue in [a merge request|https://git.mesalab.cn/tango/tsg_master/-/merge_requests/415] of [TSG Appliance / tsg_master|https://git.mesalab.cn/tango/tsg_master] on branch [feature-add-timestamp-ms-OMPUB-999-2|https://git.mesalab.cn/tango/tsg_master/-/tree/feature-add-timestamp-ms-OMPUB-999-2]:{quote}OMPUB-999: 功能端输出common_start_timestamp_ms，common_end_timestamp_ms日志字段{quote}
|
||
|
||
|
||
|
||
---
|
||
|
||
**gitlab** commented on *2023-10-13T17:05:27.945+0800*:
|
||
|
||
[刘学利|https://git.mesalab.cn/liuxueli] mentioned this issue in [a commit|https://git.mesalab.cn/tsg/tsg-os-buildimage/-/commit/ad4293bc419733bf1e6296e3cd91eedd57721925] of [TSG / tsg-os-buildimage|https://git.mesalab.cn/tsg/tsg-os-buildimage] on branch [update-v23.10-firewall-v2|https://git.mesalab.cn/tsg/tsg-os-buildimage/-/tree/update-v23.10-firewall-v2]:{quote}更新sapp、tsg_master、libosfp, 支持:{quote}
|
||
|
||
|
||
|
||
---
|
||
|
||
**gitlab** commented on *2023-10-13T17:05:55.458+0800*:
|
||
|
||
[刘学利|https://git.mesalab.cn/liuxueli] mentioned this issue in [a merge request|https://git.mesalab.cn/tsg/tsg-os-buildimage/-/merge_requests/1827] of [TSG / tsg-os-buildimage|https://git.mesalab.cn/tsg/tsg-os-buildimage] on branch [update-v23.10-firewall-v2|https://git.mesalab.cn/tsg/tsg-os-buildimage/-/tree/update-v23.10-firewall-v2]:{quote}更新sapp、tsg_master、libosfp, 支持:{quote}
|
||
|
||
|
||
|
||
---
|
||
|
||
**gitlab** commented on *2023-10-13T17:24:05.636+0800*:
|
||
|
||
[付明卫|https://git.mesalab.cn/fumingwei] mentioned this issue in [a commit|https://git.mesalab.cn/tsg/tsg-os-buildimage/-/commit/3b3d202cc68a58fc8fb9d662b325971f1163d27d] of [TSG / tsg-os-buildimage|https://git.mesalab.cn/tsg/tsg-os-buildimage] on branch [update-v23.10-firewall-v2|https://git.mesalab.cn/tsg/tsg-os-buildimage/-/tree/update-v23.10-firewall-v2]:{quote}更新sapp、tsg_master、libosfp, 支持:{quote}
|
||
|
||
|
||
|
||
---
|
||
|
||
**doufenghu** commented on *2023-10-23T13:14:03.203+0800*:
|
||
|
||
v23.07 会话开始和结束时间支持毫秒级时间戳字段存储。v23.10 支持在TSG 界面展示。
|
||
|
||
|
||
|
||
---
|
||
|
||
|
||
|
||
## Attachments
|
||
|