tencent cloud

Cloud Object Storage

DocumentaçãoCloud Object StorageData Lake StorageMetadata AcceleratorData ManagementMetadata Acceleration Bucket Inventory Feature

Metadata Acceleration Bucket Inventory Feature

PDF
Modo Foco
Tamanho da Fonte
Última atualização: 2025-11-24 11:39:34
Metadata Acceleration Bucket reuses the COS Storage Bucket Inventory feature, providing file metadata export functionality similar to the Hadoop OIV tool. The metadata acceleration bucket can perform scheduled scans of files in the specified path within the user's bucket daily or weekly according to the user's inventory task configuration, and produce an inventory report stored in CSV format in the user-specified path. It is suitable for file analysis, audit, governance, and other scenarios.
Note:
The Metadata Acceleration Bucket Inventory Feature allowlist is currently in public beta. This document is for reference during the beta test. To apply for enabling the inventory feature, please contact us.
During the public beta period, the inventory feature is temporarily free of COS management feature fees. Subsequent pricing information will be notified via internal message and official documentation.

Use Limits

Metadata acceleration bucket inventory has the following limitations during the public beta period:
Each COS metadata acceleration bucket supports a maximum of 5 configured inventory rules.
Note:
Typically, it is recommended to configure the bucket root path as the manifest scan path. If your usage scenario requires configuring more than 5 rules, please contact us.
Inventory reports do not support cross-bucket delivery. The target path in task configuration must match the source bucket.
Inventory does not support custom IDs. A task ID is automatically assigned upon task creation. This ID can be queried via the List API.
Metadata acceleration buckets do not currently support instant inventory.

Inventory Report Delivery Instructions

After configuring an inventory task, COS will perform scheduled scans of files in the specified path of your bucket based on the configuration and output an inventory report in CSV format. The delivery path for the report can be specified in the task configuration. The specific path for the CSV file is as follows:
destination-prefix/YYYY-MM-DD-HH/Data/8d1048acf48096d17502bd62efc9f8c1.csv
destination-prefix: The shipping route specified by the user in the list delivery task.
YYYY-MM-DD-HH: Timestamp, the year, month, day, and hour when the inventory task is executed.
Note:
The hour in the path indicates the task start execution time, which may differ from the actual delivery time. COS metadata acceleration buckets can only be configured for daily tasks. The initial configuration or delivery within 24 hours will generate one copy of the inventory report. It is unable to specify a fixed time to execute inventory tasks or deliver inventory reports.
Inventory report: Delivered in CSV format, with filenames as random values generated by COS.

Manifest tasks scan files under the configured path according to rule configuration. The generated report excludes header info and prints field order as follows:
Field
Description
FileName
File name.
ATime
File access time
Note:
To ensure the best read/write performance, the metadata acceleration bucket does not enable ATime tracing by default. If you need the ATime feature, contact us.
MTime
Change time of file content
CTime
Change time of file metadata
Size
File size (Byte)
FileType
Print file status as directory or file, output format: DIR (directory) or FILE (file)
ACL
Print file permissions in digits, for example: 755
UserName
Username, for example: hadoop
GroupName
User group name, for example: supergroup
StorageClass
The current file storage type. Possible values:
STANDARD (standard storage)
DEGRADE (infrequent storage)
ARCHIVE (archive storage)
DEEP_ARCHIVE (deep archive storage)
ARCHIVE+STANDARD (archive storage restored to standard)
DEEP_ARCHIVE+STANDARD (deep archive storage restored to standard)

Manifest Configuration Method

Currently, metadata acceleration buckets in COS cannot be accessed through the console for inventory configuration and only supports using APIs to configure. Specific steps are as follows:
1. Create a COS role.
2. Bind permissions to the COS role.
3. Enable the inventory feature.

1. Creating a COS role

Create a COS role. For more information about the API, see CreateRole.
Here, roleName should be COS_QcsRole.
policyDocument should be:
{
    "version": "2.0",
    "statement": [{
        "action": "name/sts:AssumeRole",
        "effect": "allow",
        "principal": {
            "service": "cos.cloud.tencent.com"
        }
    }]
}

2. Binding Permissions to the COS Role

Bind permissions to the role. For more information about the API, see AttachRolePolicy.
Here, policyName is QcloudCOSFullAccess, and roleName is the COS_QcsRole in step 1 or the roleID returned when roleName is created.

3. Enabling the Inventory Feature

Call the API to enable the inventory feature. Metadata acceleration buckets reuse the COS inventory capacity and support using the COS Inventory API to enable inventory configuration. Due to the metadata acceleration bucket feature, some fields in the API differ from the COS inventory API. For the specific calling method, please refer to the following API example:

Anterior: Metadata Acceleration Bucket LifecyclePróximo: Migrating HDFS Data to Metadata Acceleration-Enabled Bucket

Ajuda e Suporte

Esta página foi útil?

comentários