Release Notes

Product Introduction

Overview

Strengths

Use Cases

Purchase Guide

Billing Overview

Refund

Payment Overdue

Configuration Adjustment Fees

Getting Started

Complete Process for New User Activation

DLC Data Import Guide

Quick Start with Data Analytics in Data Lake Compute

Quick Start with Permission Management in Data Lake Compute

Quick Start with Partition Table

Enabling Data Optimization

Cross-Source Analysis of EMR Hive Data

Standard Engine Configuration Guide

Configuring Data Access Policy

Operation Guide

Console Operation Introduction

Development Guide

Runtime Environment

SparkJar Job Development Guide

PySpark Job Development Guide

Query Performance Optimization Guide

UDF Function Development Guide

System Restraints

Client Access

JDBC Access

TDLC Command Line Interface Tool Access

Third-party Software Linkage

Python Access

Practical Tutorial

Accessing DLC Data with Power BI

Table Creation Practice

Using Apache Airflow to Schedule DLC Engine to Submit Tasks

Direct Query of DLC Internal Storage with StarRocks

Spark cost optimization practice

DATA + AI

Using DLC to Analyze CLS Logs

Using Role SSO to Access DLC

Resource-Level Authentication Guide

Implementing Tencent Cloud TCHouse-D Read and Write Operations in DLC

DLC Native Table

SQL Statement

SuperSQL Statement

Overview of Standard Spark Statement

Overview of Standard Presto Statement

Reserved Words

API Documentation

History

Introduction

API Category

Making API Requests

Data Table APIs

Task APIs

Metadata APIs

Service Configuration APIs

Permission Management APIs

Database APIs

Data Source Connection APIs

Data Optimization APIs

Data Engine APIs

Resource Group for the Standard Engine APIs

Data Types

Error Codes

General Reference

Error Codes

Quotas and limits

Operation Guide on Connecting Third-Party Software to DLC

FAQs

FAQs on Permissions

FAQs on Engines

FAQs on Features

FAQs on Spark Jobs

DLC Policy

Data Privacy And Security Agreement

Service Level Agreement

Cross-Source Analysis of EMR Hive Data

PDF

Focus Mode

Font Size

Last updated: 2024-07-17 15:27:21

Data Lake Compute allows you to configure an EMR Hive data source for multi-source federated data analysis.
Preparations
Get the EMR Hive address.
Use an account with the permission to create data catalogs. For more information on permissions, see Permission Overview.
Creating an EMR Hive data source
1. Log in to the Data Lake Compute console and select the service region.
2. Select Data Explore on the left sidebar, click + in the Database & table column, and select Create data catalog.
﻿
﻿
3. Select EMR Hive (HDFS) for Connection type and select the target EMR instance. The VPC information will be populated by default after the instance is selected. EMR versions supported by EMR Hive are 2.3.5, 2.3.7, 3.1.1, and 3.1.2.
Note: 
Relevant permissions are required for you to select the EMR Hive instance.
﻿
4. Select the Run cluster. Currently, you can only select a private data engine of Presto. If there is no engine, create one on the Data engine page. For more information on the purchase process, see Purchasing Private Data Engine.
Note: 
The IP range of the selected data engine cannot be the same as that of the EMR instance; otherwise, a network conflict will occur, and you cannot query or analyze data.
5. Click Confirm.
Querying the EMR Hive data
After the data catalog is created, you can switch to it from the Data catalog menu on the Data Explore page.
   
﻿

At this point, you can query and analyze the data catalog with SQL statements.
Select the data engine bound when the data catalog is created and click Run to get the query result.
Note: 
You can only query the data catalog with its bound data engine. To change the bound engine, click the set icon next to the data catalog.
﻿
﻿
﻿