Steps to build a cluster environment

*How to transfer files from Mac to virtual machine: Connect the host as follows: Transfer file: scp file location and file name root@IP Address: the location where files need to be stored 1. Configure NAT network Note: change the network adapter of vm to NAT mode edit /etc/sysconfig/network-scripts/ifcfg-ens33 file Modified as BOOTPR ...

Added by brandon on Wed, 09 Mar 2022 17:00:28 +0200

Flink tutorial (25) - Flink advanced features (FlinkSQL integration Hive)

01 introduction In the previous blog, we learned Flink's File Sink. Interested students can refer to the following: Flink tutorial (01) - Flink knowledge mapFlink tutorial (02) - getting started with FlinkFlink tutorial (03) - Flink environment constructionFlink tutorial (04) - getting started with FlinkFlink tutorial (05) - simple analys ...

Added by !Mikey on Wed, 09 Mar 2022 03:29:25 +0200

Flink tutorial (07) - Flink batch streaming API (Transformation example)

01 introduction In the previous blog, we have a certain understanding of the use of Source in Flink's program model. Interested students can refer to the following: Flink tutorial (01) - Flink knowledge mapFlink tutorial (02) - getting started with FlinkFlink tutorial (03) - Flink environment constructionFlink tutorial (04) - getting star ...

Added by journeyman73 on Mon, 07 Mar 2022 10:24:51 +0200

Summary of Common Firewall Commands

Catalog 1. Overview of firewalld 2. Differences between firewalld and iptables 3. The concept of firewalld region Fourth, firewalld firewall predefined nine zones 5. Firewall Data Processing Process 6. Configuration method of Firewall 1. Runtime Configuration 2. Permanent Configuration 7. Configuration method of firewalld 8. Common fi ...

Added by fothers on Fri, 04 Mar 2022 22:18:26 +0200

Flume08: [case] Channel Selectors01: Replicating Channel Selector for multiple channels

1, Channel Selectors Next, take a look at Channel Selectors Channel Selectors include: Replicating Channel Selector and Multiplexing Channel Selector The Replicating Channel Selector is the default Channel selector. It will send the events collected by the Source to all channels Check the official documentation for an explanation of this defau ...

Added by deed02392 on Fri, 04 Mar 2022 21:08:31 +0200

Python project practice: analyze big data with PySpark

Python project practice: analyze big data with PySpark Big data, as its name implies, is a large amount of data. Generally, these data are above PB level. PB is the unit of data storage capacity, which is equal to the 50th power of 2 bytes, or about 1000 TB in value. These data are characterized by a wide variety, including video, voice, pictu ...

Added by ztealmax on Fri, 04 Mar 2022 19:19:29 +0200

Flume07: [case] Source Interceptors of flume's advanced components

1, Source Interceptors Next, let's take a look at the first advanced component, Source Interceptors Many Source Interceptors have been built into the system Common types of Source Interceptors: Timestamp Interceptor, Host Interceptor, Search and Replace Interceptor, Static Interceptor, Regex Extractor Interceptor, etc 1,Timestamp Interceptor ...

Added by bdamod1 on Fri, 04 Mar 2022 07:35:30 +0200

Common commands for practical operation

preparation Start hadoop cluster [amelia@hadoop102 hadoop-2.7.2]$ sbin/start-dfs.sh -help: output this command parameter [amelia@hadoop102 hadoop-2.7.2]$ hadoop fs -help rm Create / sanguo folder [amelia@hadoop102 hadoop-2.7.2]$ hadoop fs -mkdir /sanguo Check whether the sanguo file exists in hadoop 2. Upload -moveFromLocal: cut and ...

Added by balkan7 on Thu, 03 Mar 2022 08:08:04 +0200

Hive configuration update and delete data operations

Hive supports transaction and row level updates from version 0.14, but it is not supported by default. Some additional configurations are required. To support row level insert, update and delete, hive support transactions need to be configured. 1, Hive has the use scenario of ACID semantic transaction 1. Stream data. Many users use tools such ...

Added by ibo on Wed, 02 Mar 2022 10:11:06 +0200

Hadoop 08: introduction to HDFS recycle bin and security mode

1, Recycle bin for HDFS There is a recycle bin in our windows system. If you want to restore deleted files, you can restore them here. HDFS also has a recycle bin. HDFS will create a recycle bin directory for each user: / user / user name / Trash /, every file / directory deleted by the user on the Shell command line will enter the correspond ...

Added by phillips321 on Wed, 02 Mar 2022 02:13:11 +0200