Why are the traffic data in the console and the traffic data derived from logs inconsistent?
The traffic data derived from the byte count recorded in the EdgeResponseBytes field of the site acceleration access logs may not match the traffic data displayed on the console and the billing traffic data. The reasons are as follows:
Access logs can only record application layer data. In actual network transmission, the network traffic generated is 5-15% more than the pure application layer traffic. It consists of two parts:
Consumption by TCP/IP packet headers. In TCP/IP-based HTTP requests, each packet has a maximum of 1,500 bytes, including TCP and IP headers of 40-60 bytes, which generate traffic that cannot be counted by the application layer. The overhead of this part is approximately 3-4%.
TCP retransmission. During normal network transmission, around 3-10% of packets are lost on the Internet, and the server retransmits the lost packets. This type of traffic cannot be counted by the application layer and constitutes approximately 3-7% of the total traffic.
The monitoring data I see in Tencent Cloud Observability Platform and EdgeOne are not the same.
Data trends on Tencent Cloud Observability Platform and EdgeOne are generally consistent. However when it comes to the 1-minute granularity, the data can be slightly different. See below for details:
Tencent Cloud Observability Platform: Collect data from edge servers and aggregate the data with 1-minute granularity on the domain name level. This can guarantee the timeliness and stability. But it only provides data related to key metrics on the domain name level.
EdgeOne: Collect and analyze logs in real-time upon receiving the request, and then print out the result. It supports more metrics, such as traffic and requests by the device type and browser type. But the print-out time can be affected in case of request surges.
Assume that a user requests a 1 GB file. The download starts at 10:00:00 and ends at 10:01:40.
Tencent Cloud Observability Platform: Every edge server reports the metric data at a 1-minute interval. Data of this event is recorded at both 10:01 and 10:02.
EdgeOne: Every edge server prints a log when the download ends (10:01:40). The data is recorded at 10:01.
Therefore, data from Tencent Cloud Observability Platform and EdgeOne can differ at a 1-minute granularity due to the difference of sampling rules.