Spark13: Spark Program Performance Optimization 01: high performance serialization class library, persistence or checkpoint, JVM garbage collection tuning, improving parallelism and data localization
1, Performance optimization analysis
The execution of a computing task mainly depends on CPU, memory and bandwidth.
Spark is a memory based computing engine, so for it, the biggest impact may be memory. Generally, our tasks encounter performance bottlenecks, and most of them are memory problems. Of course, CPU and bandwidth may also affect th ...
Added by matthewst on Wed, 09 Mar 2022 04:30:50 +0200
Vue optimization quick check
Starting portal: https://ssshooter.com/2021-03...Split componentI also thought that dismantling molecular components was used for abstraction, but practice told me that dismantling molecular components was a way to improve performance (specific situation).I have encountered such a problem in my actual work. There is a large table with multiple ...
Added by chelsea7 on Thu, 24 Feb 2022 12:10:31 +0200
C + + Performance Optimization -- memory pool technology
1, Memory pool introduction
1. Introduction to C + + memory pool
Memory pool is a kind of memory allocation method. It is to apply for allocating a certain number of memory blocks of equal size (generally) for standby before using memory. When there is a new memory demand, a part of the memory block is separated from the memory pool. If the m ...
Added by Majes on Mon, 21 Feb 2022 04:26:00 +0200
Use of PureComponent for performance optimization of React components
Article from personal blog: https://knightyun.github.io/2021/05/09/js-react-purecomponent , reprint, please state
In the React class component, if the state changes, it will trigger the re rendering of the component (execute the render method), and it is all re rendering including all sub components, whether some sub components use the value ...
Added by smartknightinc on Thu, 17 Feb 2022 15:17:09 +0200
Android performance optimization memory leak, you want here~
preface
In Android, memory leakage is very common; The consequences of memory leakage will make the application Crash This article comprehensively introduces the essence, causes and solutions of memory leakage, and finally provides some common memory leakage analysis tools. I hope you will like them.
catalogue
1.png
1. Introduction
ML (M ...
Added by JeffK on Thu, 10 Feb 2022 22:51:58 +0200
Java: difference and comparison between ArrayList and LinkedList
preface
The difference between ArrayList and LinkedList is often mentioned. A clear understanding of the difference between them is helpful to consolidate their basic development skills. Choosing the right data structure in the right place can help them write more high-quality code. Combined with their source code, this paper analyzes th ...
Added by cdxrevvved on Wed, 02 Feb 2022 09:49:40 +0200
Day517. Index optimization and query optimization mysql
Index optimization and query optimization
1, Data preparation
Insert 500000 entries in the student table and 10000 entries in the class table.
Step 1: create a table
#Class table
CREATE TABLE `class` (
`id` INT(11) NOT NULL AUTO_INCREMENT,
`className` VARCHAR(30) DEFAULT NULL,
`address` VARCHAR(40) DEFAULT NULL,
`monitor` INT NULL ,
PRIMARY ...
Added by gnetcon on Wed, 26 Jan 2022 21:19:06 +0200
25, How does Linux disk I/O work?
catalogue
Disk performance index
Attention
Usage scenario
Performance testing tool fio
Disk I/O observation
Interpretation of iostat index
Process I/O observation
pidstat
iotop
Question
How to determine whether it is random io or sequential io?
Disk performance index
Utilization, saturation, IOPS, throughput, response time, etc ...
Added by jworisek on Sat, 22 Jan 2022 12:48:58 +0200
Android picture performance optimization: Bitmap
1, Introduce
As a large memory consumer, pictures have always been the key object that developers try to optimize. Bitmap's memory is located in native before 3.0, then changed to jvm, and then changed back to native after 8.0. Each jvm process has a memory limit, while native has no limit (not without impact, at least not oom). Therefore, mov ...
Added by munky334 on Sun, 16 Jan 2022 12:53:03 +0200
Spark performance optimization guide - train of thought
preface
Spark job optimization is actually a general topic, because sometimes it is slow, but the solution is really different. I want to point out all aspects of optimization so that the system can formulate the overall optimization scheme.
Sorting out optimization ideas
How to treat the so-called slow problem? I made a sorting:
themeresou ...
Added by jber on Fri, 14 Jan 2022 22:46:36 +0200