tencent cloud

Cloud Log Service

Log Distribution

Download
Mode fokus
Ukuran font
Terakhir diperbarui: 2026-05-27 21:08:02

Scenario Description

A user collects logs to CLS (Cloud Log Service). The logs are delimited by two vertical bars|| and contain fields such as log time, log level, task ID, process name, and host IP address. Now, the user wants to structure the logs and distribute them to three different target log topics according to the three levels of ERROR, WARNING, and INFO for subsequent analysis.

Scenario Analysis

Log structuring: structure the logs according to the two vertical bars|| delimiter.
Log distribution: distribute logs to three different target log topics according to three levels of ERROR, WARNING and INFO.
Note:
When creating a data processing task, select Fixed Log Topic as the target log topic and define the target name in advance. This name will be used in the log_output("target_name") function.




Raw Log

[
{
"message": "2021-12-09 11:34:28.279||INFO||605c643e29e4||BIN--Python||192.168.1.1"
},
{
"message": "2021-12-09 11:35:28.279||WARNING||615c643e22e4||BIN--Java||192.168.1.1"
},
{
"message": "2021-12-09 11:36:28.279||ERROR||635c643e22e4||BIN--Go||192.168.1.1"
}
]

DSL Processing Function

// Use `||` as the delimiter (the vertical bar needs to be escaped) to extract the time, loglevel, taskId, processName, and ip fields from the message.
ext_sepstr("message","time,loglevel,taskId,ProcessName,ip",sep="\\|\\|")

// Discard the message.
fields_drop("message")

// Use a switch statement for distribution.
t_switch(

// If the loglevel is "INFO", output to info_log.
op_str_eq(v("loglevel"),"INFO"),log_output("info_log"),
// If the loglevel is "WARNING", output to warning_log.
op_str_eq(v("loglevel"),"WARNING"),log_output("warning_log"),
// If the loglevel is "ERROR", output to error_log.
op_str_eq(v("loglevel"),"ERROR"),log_output("error_log")

)

Processing Result

In error_log:
{
"processName":"BIN--Go",
"ip":"192.168.1.1",
"loglevel":"ERROR",
"taskId":"635c643e22e4",
"time":"2021-12-09 11:36:28.279"
}
In warning_log:
{
"processName":"BIN--Java",
"ip":"192.168.1.1",
"loglevel":"WARNING",
"taskId":"615c643e22e4",
"time":"2021-12-09 11:35:28.279"
}
In info_log:
{
"processName":"BIN--PYTHON",
"ip":"192.168.1.1",
"loglevel":"INFO",
"taskId":"605c643e29e4",
"time":"2021-12-09 11:34:28.279"
}
Note:
The target log topic and target name must be configured in advance.


Bantuan dan Dukungan

Apakah halaman ini membantu?

masukan