Steps to build a cluster environment
*How to transfer files from Mac to virtual machine:
Connect the host as follows: Transfer file: scp file location and file name root@IP Address: the location where files need to be stored
1. Configure NAT network
Note: change the network adapter of vm to NAT mode
edit /etc/sysconfig/network-scripts/ifcfg-ens33 file
Modified as BOOTPR ...
Added by brandon on Wed, 09 Mar 2022 17:00:28 +0200
Flink tutorial (25) - Flink advanced features (FlinkSQL integration Hive)
01 introduction
In the previous blog, we learned Flink's File Sink. Interested students can refer to the following:
Flink tutorial (01) - Flink knowledge mapFlink tutorial (02) - getting started with FlinkFlink tutorial (03) - Flink environment constructionFlink tutorial (04) - getting started with FlinkFlink tutorial (05) - simple analys ...
Added by !Mikey on Wed, 09 Mar 2022 03:29:25 +0200
Flink tutorial (07) - Flink batch streaming API (Transformation example)
01 introduction
In the previous blog, we have a certain understanding of the use of Source in Flink's program model. Interested students can refer to the following:
Flink tutorial (01) - Flink knowledge mapFlink tutorial (02) - getting started with FlinkFlink tutorial (03) - Flink environment constructionFlink tutorial (04) - getting star ...
Added by journeyman73 on Mon, 07 Mar 2022 10:24:51 +0200
Summary of Common Firewall Commands
Catalog
1. Overview of firewalld
2. Differences between firewalld and iptables
3. The concept of firewalld region
Fourth, firewalld firewall predefined nine zones
5. Firewall Data Processing Process
6. Configuration method of Firewall
1. Runtime Configuration
2. Permanent Configuration
7. Configuration method of firewalld
8. Common fi ...
Added by fothers on Fri, 04 Mar 2022 22:18:26 +0200
Flume08: [case] Channel Selectors01: Replicating Channel Selector for multiple channels
1, Channel Selectors
Next, take a look at Channel Selectors Channel Selectors include: Replicating Channel Selector and Multiplexing Channel Selector The Replicating Channel Selector is the default Channel selector. It will send the events collected by the Source to all channels Check the official documentation for an explanation of this defau ...
Added by deed02392 on Fri, 04 Mar 2022 21:08:31 +0200
Python project practice: analyze big data with PySpark
Python project practice: analyze big data with PySpark
Big data, as its name implies, is a large amount of data. Generally, these data are above PB level. PB is the unit of data storage capacity, which is equal to the 50th power of 2 bytes, or about 1000 TB in value. These data are characterized by a wide variety, including video, voice, pictu ...
Added by ztealmax on Fri, 04 Mar 2022 19:19:29 +0200
Flume07: [case] Source Interceptors of flume's advanced components
1, Source Interceptors
Next, let's take a look at the first advanced component, Source Interceptors
Many Source Interceptors have been built into the system Common types of Source Interceptors: Timestamp Interceptor, Host Interceptor, Search and Replace Interceptor, Static Interceptor, Regex Extractor Interceptor, etc
1,Timestamp Interceptor ...
Added by bdamod1 on Fri, 04 Mar 2022 07:35:30 +0200
Common commands for practical operation
preparation
Start hadoop cluster
[amelia@hadoop102 hadoop-2.7.2]$ sbin/start-dfs.sh
-help: output this command parameter
[amelia@hadoop102 hadoop-2.7.2]$ hadoop fs -help rm
Create / sanguo folder
[amelia@hadoop102 hadoop-2.7.2]$ hadoop fs -mkdir /sanguo
Check whether the sanguo file exists in hadoop 2. Upload
-moveFromLocal: cut and ...
Added by balkan7 on Thu, 03 Mar 2022 08:08:04 +0200
Hive configuration update and delete data operations
Hive supports transaction and row level updates from version 0.14, but it is not supported by default. Some additional configurations are required. To support row level insert, update and delete, hive support transactions need to be configured.
1, Hive has the use scenario of ACID semantic transaction 1. Stream data. Many users use tools such ...
Added by ibo on Wed, 02 Mar 2022 10:11:06 +0200
Hadoop 08: introduction to HDFS recycle bin and security mode
1, Recycle bin for HDFS
There is a recycle bin in our windows system. If you want to restore deleted files, you can restore them here. HDFS also has a recycle bin.
HDFS will create a recycle bin directory for each user: / user / user name / Trash /, every file / directory deleted by the user on the Shell command line will enter the correspond ...
Added by phillips321 on Wed, 02 Mar 2022 02:13:11 +0200