Programming VIP

1. Calling by name summary The function will be called only when the return value of the function is really used example Normal mode call code   def main(args: Array[String]): Unit = {     f2(f1())   }   def f1(): Int ={     println("f1")     return 1   }   def f2(a:Int): Unit ={     println("f2")     println(a)   }   - Output results Outp ...

1. if else 1.1 single branch Syntax structure: if (expr) { expr by true Statement executed when } 1.2 double branch Syntax structure: if (expr) { expr by true Statement executed when } else { expr by false Statement executed when } 1.3 multi branch Syntax structure: if (expr1) { expr1 by true Statement executed when } ...

1 replication and multiplexing 1.1 case requirements Flume-1 is used to monitor file changes. Flume-1 passes the changes to Flume-2, which is responsible for storing them To HDFS. At the same time, Flume-1 passes the changes to Flume-3, which is responsible for outputting them to the local file system. 1.2 demand analysis: single data ...

1. Process control structure 1.1 general In the actual development, we have to write thousands of lines of code. The order of the code is different, and the execution results will certainly be affected. Some codes can be executed only if they meet specific conditions, and some codes need to be executed repeatedly. How to reasonably plan these ...

1. RDD persistence operation Acts on the behavior of the operator RDD does not store data. If an RDD needs to be reused, it needs to be executed again from the beginning to obtain data RDD objects can be reused, but data cannot be reused By default, it is saved in memory. There is a parameter in persist (default memory) Put it in memory: save ...

1. Spark overview 1.1 what is Spark Spark is a fast, universal and scalable big data analysis framework based on memory. 1.2 Hadoop and Spark Hadoop: a one-time computing framework based on disk, which is not suitable for iterative computing. When processing data, the framework will flush the storage device to read out the data, carry out ...

Detailed explanation of lambda expression in Java Lambda expressions, also known as closures, are the most important new feature that drives the release of Java 8. Lambda allows functions to be used as parameters of a method (functions are passed into the method as parameters). Using Lambda expressions can make the code more concise and co ...

Spark Doris Connector is a new feature introduced by Doris in version 0.12. Users can use this function to directly read and write the data stored in Doris through spark, and support SQL, Dataframe, RDD and other methods. From the perspective of Doris, introducing its data into Spark can use a series of rich ecological products of Spark, bro ...

Catalog Data Collection Type Invariant Array Variable Array Invariant List Variable List Invariant Set Variable Set Invariant Map Variable Map tuple Data Collection Type scala's data collection types are mainly Array, List, Set, Map, Tuple, etc. They are divided into variable and immutable collections, which can be updated and ...

Java syntax sugar Syntax sugar, also known as sugar coated grammar, is a term invented by British computer scientist Peter J. landin. It refers to a grammar added to the computer language. This grammar has no impact on the function of the language, but it is more convenient for programmers to use. The most commonly used syntax sugars in Java m ...