MapReduce actual case, MapTask operation mechanism, ReduceTask operation mechanism, MapReduce execution process, hadoop data compression, implementation of Join algorithm
MapReduce actual case, MapTask operation mechanism, ReduceTask operation mechanism, MapReduce execution process, hadoop data compression, implementation of Join algorithm
MapReduce actual case
Reverse order of upstream traffic
Cell phone number division
MapTask operation mechanism
Operation proces ...
Added by Fergal Andrews on Sat, 13 Jun 2020 09:39:07 +0300
Implement simple chat function 1 to 1 based on netty (private message / group sending)
netty chat
explain:
My thoughts
demonstration
Chat server
Chat initialization configuration
Client processing message
Front end code
epilogue
explain:
I found that it takes a lot of energy to write a blog. In order to leave more time for promotion, I just post the code. Learning address: https: ...
Added by toppac on Sun, 07 Jun 2020 07:35:12 +0300
logstash and filebeat collect logs
1. Background
Log collection adopts ELK framework, i.e. elasticsearch, logstash, kibana, and filebeat component. Filebeat is used to scan log files and send logs to logstash service, while logstash service completes log segmentation and sends logs to elasticsearch service.
2. Deployment steps of filebeat (log - > logstash)
Download the ...
Added by Funbruk on Sun, 31 May 2020 19:13:13 +0300
Consumption mode of consumer of pulsar
client processing logic
ConnectionPool
netty related configurations are available in com.yahoo.pulsar.client.impl.ConnectionPool
public ConnectionPool(final PulsarClientImpl client, EventLoopGroup eventLoopGroup) {
this.eventLoopGroup = eventLoopGroup;
this.maxConnectionsPerHosts = client.getConfiguration().getConnection ...
Added by Nothsa on Wed, 27 May 2020 14:04:46 +0300
Self timer 66 Python ffmpeg bulk compression video
Case story: Bug videos are found during the test, which need to be provided for reference in the development of Bug solution,
However, after video shooting, it is too large to be transmitted on wechat client,
Therefore, there have been testers using winzip to compress in batches (part1, part2, part3),
Then transmit the "chaos" of vid ...
Added by ciaranmg on Tue, 19 May 2020 20:11:39 +0300
Soft and hard decoding implementation of android ffmpeg (ffmpeg 3.3.4)
1. Soft decoding implementation:
JNIEXPORT int JNICALL Java_h264_Native_PlayLocalVideo(JNIEnv *env, jobject obj,jstring inputFilePath_,jobject surface) {
const char *path = env->GetStringUTFChars(inputFilePath_, 0);
av_log_set_callback(ffmpeg_android_log_callback);
av_register_all();
int ret;
AVFormatContext *fmt_ctx ...
Added by ericorx on Fri, 01 May 2020 21:12:02 +0300
A summary of Jmeter 's Rsa encryption and decryption
Encrypt script first:
import org.apache.commons.codec.binary.Base64;
import java.io.ByteArrayOutputStream;
import java.security.Key;
import java.security.KeyFactory;
import java.security.KeyPair;
import java.security.KeyPairGenerator;
import java.security.PrivateKey;
import java.security.PublicKey;
import java.security.Signature;
i ...
Added by kind on Thu, 09 Apr 2020 17:46:55 +0300
Dubbo Source Learning-Service Reference (Remote Call Phase)
In the previous article, we looked at what was done during the startup phase of a Dubbo service reference, and what was not seen during the remote invocation phase.The Dubbo service invocation process is complex and involves many steps, such as sending requests, codec, service demotion, filter chain processing, serialization, thread dispatch, ...
Added by Aaron111 on Sat, 21 Mar 2020 05:19:50 +0200
Deep in tutorial: FFmpeg, convert MP4 and other video files to gif dynamic pictures
Video files such as MP4 are transformed into gif dynamic pictures
Video files such as MP4 are transformed into gif dynamic pictures
FFmpeg installation
FFmpeg easy to use
convert installation
convert easy to use
Video files such as MP4 are transformed into gif dynamic pictures
ImageMagick image ...
Added by celsoendo on Sat, 07 Mar 2020 11:53:30 +0200
Summary of Basic Instructions for Installing Kafka in CDH - topic uses and tests producer to generate data, consumer consumption data
Here is a summary of some of the more basic instructions for the kafka test I installed in the CDH.
1. Related Basic Contents
Each host in the Kafka cluster runs a server called a proxy that stores messages sent to the subject and serves consumer requests.
First, look at the instance information o ...
Added by bapi on Sat, 22 Feb 2020 05:00:13 +0200