Scenarios
Cloud Log Service (CLS) manages Log data via Log topics and logsets. A Log Topic is the basic unit for collecting, storing, retrieving, and analyzing Log data, while a logset classifies Log topics to help users manage them.
This article introduces how to manage log topics through console management. Before actual operation, it is advisable to read Log Topics and Logsets to further understand the relevant concepts and usage recommendations for actual scenarios. Prerequisites
Operation Steps
Log Topic Creation
1. In the left sidebar, click Log Topic.
2. On the log topic management page, select the region of the log topic, and click Create Log Topic.
3. In the pop-up window, fill in the following information:
Log Topic name: such as nginx, support 1 - 255 characters.
Log TopicID: a random ID is generated by default; you can also define an ID. Only lowercase letters, numbers, and - are supported, and it cannot start or end with -. Length 3 to 40 characters.
Log retention policy:
Storage Option: support enabling permanent storage and log settlement feature. Storage Period: Defaults to 30 days. You can customize log storage time. Logs will be automatically cleared after expiration. When using LogListener, if log time does not use collection time but a time field in the log content as the log time, that time will be used to determine whether the log expires.
Logset Operation:
Select an existing logset: select a target logset from the Logset dropdown list.
Create Logset: Enter the name of the new logset, and set a tag for the current logset to be created so that resources can be managed by category from different dimensions.
Log Topic tag: Set a Tag for the current log topic being created, making it easier to manage resources from different dimensions.
Advanced Settings:
Anonymous upload: Once enabled, designated operations will be supported to anonymously upload the log topic.
Anonymous operation: After enabling anonymous upload, currently supports configuring the following operations for anonymous upload to the log topic:
JS/HTTP log upload: Password-free log upload to this log topic via JS, HTTP. For details, see anonymous log upload. Kafka Protocol Log Upload: Password-free log upload to this log topic via Kafka protocol. For details, see anonymous log upload. Filters: When anonymous upload is enabled, only requests that meet specific conditions are allowed to anonymously upload to this log topic. Currently supports configuring the following filter criteria:
VPC_ID: The source VPC of the request. Only Tencent Cloud VPC is supported, and the VPC region must match the current log topic.
Note:
Currently, filter criteria can be configured only for logs uploaded anonymously via the Kafka protocol.
Record Public IP Address and Log Arrival Time: When this feature is enabled, the public IP address and the time the log arrives at the server are automatically added as metadata fields.
Public IP address: This field records the public IP address of the device from which the log originates. After you enable this option, CLS automatically adds this field to the collected log data, and the field name is __TAG__.client_ip.
Log Arrival Time: This field records the time when a log arrives at the CLS server, using a 10-digit Unix timestamp format (second-level precision). After you enable this option, CLS automatically adds this field to the collected log data, and the field name is __TAG__.receive_time.
Log Topic description: Description information to facilitate understanding the use of this log topic. Select existing log set: Choose the target logset from the Logset dropdown list.
Initial partitions: the number of partitions automatically created when the topic is created, supporting 1-10, default is 1. For more information on topic partitions, see Topic Partition. Auto-split: whether to automatically add partitions based on the topic's write traffic size, enabled by default.
Maximum partitions: When auto split is enabled, the maximum number of partitions is 50 by default.
4. Click OK to create a log topic.
After the log topic is successfully created, you can click the log topic ID/name on the log management page to view its details.
Note:
You are advised to select the same region as CVM or any other cloud product from which you collect logs.
After the log topic is created, its region, logset, and access storage type cannot be modified.
Editing Log Topic
1. In the left sidebar, click Log Topic.
2. On the log topic management page, find the log topic ID/name you need to edit, then click Edit on the right side of the list.
3. In the pop-up window, modify the basic information of the log topic.
4. Click OK to complete the update of the log topic information.
Deleting a Log Topic
1. In the left sidebar, click Log Topic.
2. On the log topic management page, find the log topic ID/name to be deleted, click More on the right side of the list to open the operation menu and select Delete Log Topic.
Note:
You can switch to different regions in the upper-left corner of the page to delete log topics within each region. To delete all log topics, it is recommended to use an account with full resource permissions; otherwise, some log topics may not be fully deleted as you do not have permissions to view them.
3. In the pop-up window, check the resources associated with the log topic. If there are data processing tasks or scheduled SQL tasks, please delete the corresponding resources first, then click Next and click Delete to delete the current log topic.
Note:
Once a log topic is deleted, its configurations and log data cannot be recovered. Proceed with caution.