tencent cloud

Cloud Log Service

Release Notes and Announcements
Release Notes
Announcements
User Guide
Product Introduction
Overview
Features
Available Regions
Limits
Concepts
Service Regions and Service Providers
Purchase Guide
Billing Overview
Product Pricing
Pay-as-You-Go
Billing
Cleaning up CLS resources
Cost Optimization
FAQs
Getting Started
Getting Started in 1 Minute
Getting Started Guide
Quickly Trying out CLS with Demo
Operation Guide
Resource Management
Permission Management
Log Collection
Metric Collection
Log Storage
Metric Storage
Search and Analysis (Log Topic)
Search and Analysis (Metric Topic)
Dashboard
Data Processing documents
Shipping and Consumption
Monitoring Alarm
Cloud Insight
Independent DataSight console
Historical Documentation
Practical Tutorial
Log Collection
Search and Analysis
Dashboard
Monitoring Alarm
Shipping and Consumption
Cost Optimization
Developer Guide
Embedding CLS Console
CLS Connection to Grafana
API Documentation
History
Introduction
API Category
Making API Requests
Topic Management APIs
Log Set Management APIs
Index APIs
Topic Partition APIs
Machine Group APIs
Collection Configuration APIs
Log APIs
Metric APIs
Alarm Policy APIs
Data Processing APIs
Kafka Protocol Consumption APIs
CKafka Shipping Task APIs
Kafka Data Subscription APIs
COS Shipping Task APIs
SCF Delivery Task APIs
Scheduled SQL Analysis APIs
COS Data Import Task APIs
Data Types
Error Codes
FAQs
Health Check
Collection
Log Search
Others
CLS Service Level Agreement
CLS Policy
Privacy Policy
Data Processing And Security Agreement
Contact Us
Glossary

Shipping and Consumption

PDF
Focus Mode
Font Size
Last updated: 2024-12-17 17:57:09
The restrictions are shown in the table below:
Type
Restriction Item
Description
Ship to COS
Number of Tasks Submitted
For each log topic, a maximum of 10 shipping to COS tasks can be configured.
Deliverable Data Scope
Data (including historical and real-time data) of log topics in the lifecycle.
Task Submission Delay
The latency of shipping to COS is about 60 seconds.
Note:
If you set the shipping interval to be 5 minutes, logs from 12:00 to 12:05, meeting the interval of 5 minutes, will be shipped as a whole to COS in a batch. With a latency of 60 seconds, this batch of logs will arrive at COS around 12:06.
Trigger Delivery
One shipping will be triggered when a partition meets any of the following conditions:
Size of logs to be shipped (accumulated log data). Range: 5 to 256 MB (uncompressed).
Shipping interval. Range: 5 to 15 minutes.
Cross-account/Cross-region
Cross-account: supported
Cross-region: not supported
Ship to CKafka
Number of Tasks Submitted
For each log topic, one shipping to CKafka task can be configured.
Deliverable Data Scope
At present, only real-time log data can be shipped, and historical data is not supported.
Task Submission Delay
The latency of shipping 99.9% of logs to the CKafka are within 5 seconds. (The target CKafka functions properly, has a throughput capacity sufficient to process messages from the CLS and the disk is not full or throttled).
Note:
If the timestamp (value of _TIMESTAMP_) of a log collected to CLS is 12:00:00, the log will arrive at CKafka before 12:00:05.
Cross-account/Cross-region
Cross-account: supported
Cross-region: not supported
Consumption over Kafka
Versions
Kafka protocol version supported: 1.1.1 and earlier.
Number of consumer groups
Each log topic can be configured with 20 consumer groups.
Authentication Method
Only SASL_PLAINTEXT is supported.
Consumable data range
Once this feature is enabled, consumption will start from the latest data. Data will be retained for 2 hours. If data is not consumed within 2 hours, it will not be saved.
Note:
For example, if you enable consumption over Kafka protocol at 12:00, you will consume data from 12:00. Until 15:00, you can consume data from 13:00 to 15:00. Data before 13:00 cannot be consumed.
Consumption latency
The consumption latency of 99.9% log data is within 5 seconds.
Note:
If the timestamp (value of _TIMESTAMP_) of a log collected to CLS is 12:00:00, the log will arrive at the consumer before 12:00:05.
Cross-account/Cross-region
Cross-account: Account A can be a collaborator of account B to consume the log topics of account B. Permissions need to be set for the collaborator. For details, see the permission configuration.
Cross-region: Cross-region consumption can be implemented through public network.

Help and Support

Was this page helpful?

Help us improve! Rate your documentation experience in 5 mins.

Feedback