CDH6.2. The whole process of brainless construction and configuration (Beginner's version)
The software download link is at the bottom
thank: CSDN Daniel: Travel through IT bilibili Daniel: amoscloud2013
1. Preliminary preparation
Five 8G virtual machines are CDH1, cdh2, cdh3, CDH4 and cdh5 respectively. JDK is installed on all virtual machines
2. Modify IP and host name
Select CentOS 7 for cluster deployment. All three vir ...
Added by thefollower on Mon, 27 Dec 2021 05:46:50 +0200
Are you still using group by? Experts are using grouping sets
Let's take a look at a problem in actual generation:
There is a user access record table, which records the user's access under different channels, operating systems and versions, As follows:
create table test.test10 as
select '10001' as uid,'xiaomi' as qid,'android' as os,'1.2.2' as ver
union all
select '10002' as uid,'xiaomi' as qid,'andro ...
Added by jpowermacg4 on Sat, 25 Dec 2021 18:29:07 +0200
Spark sql learning notes -- DataFrame, Dataset and sql parsing principles
catalogue
1, SparkSession, DataFrame and Dataset
2, Spark Sql parsing
1. Overall overview
2. sql syntax parsing key objects
3, Spark LogicalPlan
1. Overall overview
2. LogicalPlan class structure system
3. Generated by analyzed logicalplan
1, SparkSession, DataFrame and Dataset
1. To use the sparksql function, you need to create a ...
Added by cute_girl on Fri, 24 Dec 2021 03:44:53 +0200
DDL and DML data operations of Hive
catalogue
1, DDL(Data Definition Language)
1. Create database
2. Query database
3. Modify database
4. Delete database
5. Create table
Management table (internal table)
External table
Conversion between two tables
Modify table
Delete table
Clear table
2, DML data operation
1. Data import
Load data into a table (load)
Insert data ...
Added by jtapoling on Thu, 23 Dec 2021 19:57:52 +0200
hiveSQL advanced -- 10__ Tests whether a value exists in a set of data
0 - problem description
SQL tests whether there is a value in a group of data. You want to generate a Boolean value according to whether a group of row records contain a specific value. Imagine an example where a student will take several exams over a period of time. Suppose he takes three exams every three months. As long as he has pas ...
Added by Satria Ox41464b on Thu, 16 Dec 2021 13:58:28 +0200
insert overwrite implementation of Doris partition bucket table
insert overwrite implementation of Doris partition bucket table
(1) Real time project If it is real-time, we should pay attention to timeliness. Directly select the unique key model table and select the non repeatable columns as unique keys
A simple example of creating a single partition table
CREATE TABLE order_tab
(
orderid BIGINT, ...
Added by snipesh0tz on Tue, 14 Dec 2021 10:25:54 +0200
scala -- process control + yield derivation + scala does not have continue or break?
1. Process control structure
1.1 general
In the actual development, we have to write thousands of lines of code. The order of the code is different, and the execution results will certainly be affected. Some codes can be executed only if they meet specific conditions, and some codes need to be executed repeatedly. How to reasonably plan these ...
Added by xmanofsteel69 on Fri, 26 Nov 2021 02:52:49 +0200
Schematic sequence table + single chain table
I. diagram sequence table
Concept of sequence table
Sequential table is a linear structure in which data elements are stored in sequence with a storage unit with continuous physical addresses. Generally, array storage is used. Complete the addition, deletion, query and modification of data on the array. (what we want to describe below is a dy ...
Added by waldo on Sun, 07 Nov 2021 02:56:59 +0200
Introduction and basic usage of Redis's five basic data types
Redis five data types
Redis key basic command
set key name -- Set a basic key value pair
keys * -- View all keys
exists key -- View the corresponding key Does it exist
move key Number of corresponding database -- Put this key Move the corresponding data to another database
expire key Expiration time(Unit second) -- Set how long this d ...
Added by colake on Tue, 21 Sep 2021 14:39:22 +0300