Programming VIP

Allocation of partition replicas       In the previous section, the distribution of partitioned copies is mentioned many times. Readers may be a little confused. There is also the concept of partitioned distribution among producers and consumers. The partition allocation of producers refers to specifying the partition to which e ...

1, TICDC In the previous article, we introduced how to use TICDC to synchronize data to Mysql. From the previous task, we can see that TICDC is much simpler in configuration than tidb binlog, and we also know that the performance of TICDC is also better than tidb binlog. Today, we learn how to use TICDC to synchronize data to downstream Kafka ...

Re equalizer       Rebalancing refers to the behavior that the ownership of the partition is transferred from one consumer to another. It provides a guarantee for the high availability and scalability of the consumer group, so that we can easily and safely delete consumers in the consumer group or add consumers to the consumer ...

Store for RocketMQ Source Analysis Think Before Reading Store Source When the number of topics increases to 100+, the TPS of kafka's single broker decreases by one order of magnitude, while RocketMQ remains high in the mass topic scenario? kafka has tens of thousands of TPS with less topic tps, but more topic will decrease to 1-2,000 TPS. Bu ...

Consumer client development         After understanding the concept of consumer and consumer group, we can start the development of consumer client. In the history of Kafka, the consumer client, like the producer client, has experienced two major versions: the first is a client written in scala at the beginning of Kafka's ...

1, Prepare Ensure that JDK and zookeeper services have been built on the server; If the construction is not completed, please refer to the following articles: Installing zookeeper: https://blog.csdn.net/xuan_lu/article/details/120474451 Installation of JDK1.8: https://blog.csdn.net/xuan_lu/article/details/107297710 kafka official website: ...

This article is about Learning Guide for Big Data Specialists from Zero (Full Upgrade) Kafka: Partial supplement. 1 Producer API 1.1 Message Sending Process Kafka's roducer sends messages asynchronously, involving two threads, the main thread and the Ender thread, and a thread-sharing variable, RecordAccumulator. The main thread sends mes ...

Background introduction This article will introduce the case of how to import the data in TiDB into Kafka through TiCDC and then consumed by Flink. In order to quickly verify the functionality of the whole process, all components are deployed in a single machine. If you need to deploy in a production environment, it is recommended to replace ...

Article catalog background Example explanation Construct the StreamTableEnvironment object Register table Using Tuple Using Row Pojo class using java Use external storage background sql，Structured Query Language: as a general and popular query language, Structured Query Language is becomin ...

Article catalog 1. Common configuration 2. kafka-clients 2.1 producers 2.2 consumers 2.3 multithreading 1. Common configuration producer to configure describe type Default bootstrap.servers It is used to establish the connection with kafka cluster and only affects the hosts used for init ...