Cost Saving Suggestions
When considering cost saving for CLS, pay more attention to billable items that are more costly. Index traffic and index storage are usually several times more expensive than write traffic and log storage, mainly because they are billed according to the volume of uncompressed log data. If LogListener is used to upload logs, data is compressed. Write traffic and log storage are calculated based on the volume of the compressed log data. Generally, the log compression ratio is 1:4 to 1:10. You can adjust log topic configuration as follows to save costs:
1. Reduce the amount of log data to be uploaded: Most billable items, including index traffic and index storage, are related to the amount of uploaded log data. Reducing the amount of uploaded log data can save costs at the source. You can reduce unnecessary log collection or use the filter in the LogListener collection configuration to collect only logs that meet the filter rules, for example, only Error and Warning logs.
2. Shorten the log storage period: The longer the log storage period, the higher the log storage expense and index storage expense. For older logs, if they are no longer needed for daily search and analysis and only need to be backed up as historical data, they can be shipped to COS. 3. Simplify index configuration: Simplifying index configuration can reduce index traffic and index storage costs. Note that if you enable both full-text index and key-value index, the index traffic will not be billed twice, but billed in a union-like manner. Therefore, when full-text index is enabled, reducing the number of key-value index fields does not reduce index traffic or index storage. If you do not need full-text search, you can disable full-text index and reduce unwanted key-value index fields.
4. Use infrequent access (IA) storage: IA storage is a low-cost offline log storage solution to search and store massive infrequently accessed logs. It applies to scenarios where users do not have real-time log search requirements and logs need to be stored for a long time. The solution can save about 80% of costs. For more information, see here. 5. Distribute log topics using the data processing feature: With the data processing feature, you can ship raw logs to different log topics. For example, you can classify logs by log level, ERROR, WARNING, and INFO, and then ship them to different log topics. You can create a source log topic for receiving all log data (you can disable index for the log topic so that it does not incur any index traffic or index storage fees, reducing costs), and then use the data processing feature to ship raw logs to log topics with different storage periods, different index configurations, or different storage types according to log search and analysis requirements to better balance log usage requirements and costs. For more information, see Creating a Processing Task.