Overview
During full data loading, a certain amount of resource consumption will be imposed on the primary instance of TencentDB for MySQL, especially when you perform one-time loading on instances with large data volumes. For instances with limited resources, this operation may affect the performance and stability of the primary instance to some extent. You can use the following methods to control the impact on the primary instance of TencentDB for MySQL during the data loading process and reduce resource consumption on the source side. However, these operations may reduce the efficiency of converting row-based data to columnar storage. Therefore, make adjustments based on your actual business needs.
Prerequisites
The instance is running.
Controlling Data Loading Speed
Adjust the number of concurrent data reads for a single table in the scenario of full data loading.
Modify the replication_dumper_select_parallel parameter through the feature of adjusting parameters. This parameter indicates the concurrency for simultaneous data reads from the same object. A larger value allows for a broader range of data reads from the table. In scenarios with sufficient resources, this can improve the loading efficiency. If resources are insufficient, it is recommended to set this value smaller. See Setting Instance Parameters for operation details. Adjust the number of rows read from a single table in the scenario of full data loading.
Modify the replication_dumper_chunk_size parameter through the feature of adjusting parameters. This parameter indicates the number of data rows read in a single operation for the same object. A larger value allows for more data reads each time. In scenarios with sufficient resources, this can improve the loading efficiency. If resources are insufficient, it is recommended to set this value smaller. See Setting Instance Parameters for operation details. Adjust the total number of concurrent data reads in the scenario of full data loading.
Modify the replication_dumper_table_parallel parameter through the feature of adjusting parameters. This parameter indicates the number of tables from which data can be read simultaneously. A larger value allows more tables to be read simultaneously. In scenarios with sufficient resources, this can improve the loading efficiency. See Setting Instance Parameters for operation details.