Programming VIP

Storage Based Authorization in the Metastore Server Based on storage authorization, metadata in the Metastore can be protected, but more fine-grained access control (such as column level and row level) is not providedSQL Standards Based Authorization in HiveServer2 Hive authorization based on SQL standard is fully compatible with SQL auth ...

Hive tuning: Choosing the appropriate "storage format" and "compression method" for the analyzed data can improve the analysis efficiency of hive Data compression format: When selecting a compression algorithm, you need to consider whether it can be divided, If segmentation is not supported (the integrity of a pi ...

4 - website log analysis cases - log data statistical analysis 1, Environment preparation and data import 1. Start hadoop If it is enabled in a virtual environment such as lsn, you need to perform formatting first hadoop namenode -format Start Hadoop start-dfs.sh start-yarn.sh Check to see if it starts jps 2. Import data Upload ...

26 data analysis cases -- the second stop: Civil Aviation Customer Value Analysis Based on Hive Environment required for experiment • Python: Python 3.x； • Hadoop2.7.2 environment; • Hive2.2.0 Experimental background People choose more and more travel modes, such as aircraft, high-speed rail, cars, ships, etc. in particular, aircraft ...

[software engineering practice] Hive research - Blog13 2021SC@SDUSC Research content introduction I am responsible for converting the query block QB into a logical query plan (OP Tree) The following code is from apaceh-hive-3.1 2-Src / QL / SRC / Java / org / Apache / Hadoop / hive / QL / plan, which is my analysis object code. In Blog9-12, ...

The previous chapter introduced how to install hive and the basic introduction of hive. Here we start using hive. Before using, first introduce the learning of the basic statements of hive, and what are internal tables and external tables. hive base statement Let's take a look at the most basic formats, because there are many kinds of fo ...

Let's take a look at a problem in actual generation: There is a user access record table, which records the user's access under different channels, operating systems and versions, As follows: create table test.test10 as select '10001' as uid,'xiaomi' as qid,'android' as os,'1.2.2' as ver union all select '10002' as uid,'xiaomi' as qid,'andro ...

82.1 problems When the INSERT... SELECT statement inserts data into a table in Parquet or ORC format, dynamic partitioning is enabled and cannot be executed normally Hive client: Task with the most failures(4): Diagnostic Messages for this Task: Error: GC overhead limit exceeded ... FAILED: Execution Error, return code 2 from org.apache.hadoo ...

**Requirements: you want to maintain two wide tables on phoenix, one for ad hoc query, one for one day's data and one for history. The characteristic of wide table is that it is composed of multiple tables, but the arrival time of each table is inconsistent. Some tables arrive first, and some tables may arrive every other day. What you want to ...

Reference article: https://www.cnblogs.com/qingyunzong/p/8746159.html 1, Composite data type 1,array: ARRAY<data_type> 2,map: MAP<primitive_type, data_type> 3,struct: STRUCT<col_name:data_type> The available data are as follows: 1 huangbo guangzhou,xianggang,shenzhen a1:30,a2:20,a3:100 beijing,112233, ...