Programming VIP

The previous chapter introduced how to install hive and the basic introduction of hive. Here we start using hive. Before using, first introduce the learning of the basic statements of hive, and what are internal tables and external tables. hive base statement Let's take a look at the most basic formats, because there are many kinds of fo ...

The software download link is at the bottom thank: CSDN Daniel: Travel through IT bilibili Daniel: amoscloud2013 1. Preliminary preparation Five 8G virtual machines are CDH1, cdh2, cdh3, CDH4 and cdh5 respectively. JDK is installed on all virtual machines 2. Modify IP and host name Select CentOS 7 for cluster deployment. All three vir ...

hadoop_ Hdfs07 hdfsha cluster configuration & ZK cluster configuration & yarnha configuration Note: notes (1) Cluster planning Hadoop102Hadoop03Hadoop04ZKZKZKJournaleNodeJournaleNodeJournaleNodeNameNodeNameNodeDataNodeDataNodeDataNodeResourceManagerResourceManagerNodeManagerNodeManagerNodeManager (2) Configuring Zookeeper clusters ...

Let's take a look at a problem in actual generation: There is a user access record table, which records the user's access under different channels, operating systems and versions, As follows: create table test.test10 as select '10001' as uid,'xiaomi' as qid,'android' as os,'1.2.2' as ver union all select '10002' as uid,'xiaomi' as qid,'andro ...

82.1 problems When the INSERT... SELECT statement inserts data into a table in Parquet or ORC format, dynamic partitioning is enabled and cannot be executed normally Hive client: Task with the most failures(4): Diagnostic Messages for this Task: Error: GC overhead limit exceeded ... FAILED: Execution Error, return code 2 from org.apache.hadoo ...

Building Hadoop distributed platform in linux system First, if the liunx network cannot be connected, click "Edit" in the VMwvare main interface, and then select "virtual network editor" After entering, restore the default settings in the following two steps. Generally, you can restore them after setting 1. Environmental ...

background With the growing size of HDFS clusters, It is inevitable that there will be some "slow nodes" in the cluster, which are mainly manifested in slow network data transmission and slow disk reading and writing. Usually, these slow nodes are difficult to be found. Only when the business job data reading and writing involves the ...

This blog is mainly about troubleshooting hadoop hdfs, including NameNode fault handling, cluster security mode and disk repair. If there is something bad, welcome everyone! thank! nn Fault Handling 1. Scene The NameNode process hangs and the stored data is lost. How to recover the NameNode 2. Fault simulation (1) kill -9 NameNode proce ...

Software packages required during deployment Link: https://pan.baidu.com/s/1UajMORVvQ_VSLOdVkJWYQQ Extraction code: e28y Link: https://pan.baidu.com/s/1dMj8JEaRIOaXP53W2kF_mQ Extraction code: xbyo a key: Sets the host name in FQDN formatTurn off firewallTurn off ipv6Configure local http serviceConfigure local storageThe mysql jdbc drive ...

To see which compression algorithms hadoop has [lqs@bdc112 hadoop-3.1.3]$ bin/hadoop checknative 2021-12-15 16:20:12,342 INFO bzip2.Bzip2Factory: Successfully loaded & initialized native-bzip2 library system-native 2021-12-15 16:20:12,345 INFO zlib.ZlibFactory: Successfully loaded & initialized native-zlib library 2021-12-15 16:20:12,3 ...