Programming VIP

preface    it's best to recompile the source code when building Hadoop, because some functions of Hadoop must coordinate Java class files and library files generated by Native code through JNT. To run Native code in linux system, first compile Native into [. so] file of target CPU architecture. Different processor architectures n ...

Flume log collection 1, Flume introduction Logo refers to the transfer of wood (data) from one place (data source) to another place (data destination) through the river channel (channel) Working with documents 2, Installation configuration of Flume 1. Download and upload to the server 2. Decompression 3. Configuration Copy fl ...

1, Environment introduction Install the Ubuntu virtual machine using VirtualBox. Install Hadoop and Eclipse 3.0 in Ubuntu 8 compiler. Download and install JAVA environment, Download jdk and complete the pseudo distributed environment configuration of Hadoop. Import all the required JAR packages encountered by the compiler in Eclipse. Start Had ...

6 - click stream data analysis project - log collection to HDFS reference resources: https://blog.csdn.net/tianjun2012/article/details/62424486 The basic information about logs has been introduced in the previous section. It will not be explained in detail here. Only the basic methods of generating logs and collecting logs are provided. ...

The software download link is at the bottom thank: CSDN Daniel: Travel through IT bilibili Daniel: amoscloud2013 1. Preliminary preparation Five 8G virtual machines are CDH1, cdh2, cdh3, CDH4 and cdh5 respectively. JDK is installed on all virtual machines 2. Modify IP and host name Select CentOS 7 for cluster deployment. All three vir ...

hadoop_ Hdfs07 hdfsha cluster configuration & ZK cluster configuration & yarnha configuration Note: notes (1) Cluster planning Hadoop102Hadoop03Hadoop04ZKZKZKJournaleNodeJournaleNodeJournaleNodeNameNodeNameNodeDataNodeDataNodeDataNodeResourceManagerResourceManagerNodeManagerNodeManagerNodeManager (2) Configuring Zookeeper clusters ...

background With the growing size of HDFS clusters, It is inevitable that there will be some "slow nodes" in the cluster, which are mainly manifested in slow network data transmission and slow disk reading and writing. Usually, these slow nodes are difficult to be found. Only when the business job data reading and writing involves the ...

This blog is mainly about troubleshooting hadoop hdfs, including NameNode fault handling, cluster security mode and disk repair. If there is something bad, welcome everyone! thank! nn Fault Handling 1. Scene The NameNode process hangs and the stored data is lost. How to recover the NameNode 2. Fault simulation (1) kill -9 NameNode proce ...

Flink Core Programming 1,Environment When Flink Job submits to perform calculations, it first establishes a link with the Flink framework, that is, the current Flink runtime environment in which task can be scheduled to a different taskManager execution only if environmental information is available. This environment object is relatively simp ...

0-Preface What does HDFS Secondary NameNode do? This is a classic basic interview question, and the interviewer has asked the interviewer many times (and of course many times). From the impression, about half of the interviewees can't answer correctly, and even give the answer "is not NameNode's hobby". In order to save spa ...