DataX tutorial (04) - complete interpretation of configuration

01 introduction Through the previous blog posts, we know the concept and principle of DataX: DataX tutorial (01) - getting startedDataX tutorial (02) - complete process of running dataX in IDEA (filling all pits)DataX tutorial (03) - source code interpretation (super detailed version) This article needs to explain the configuration of Da ...

Added by hatching on Fri, 11 Feb 2022 08:35:25 +0200

kettle data synchronization perfect version

Perfect version of kettle to realize data incremental synchronization preface Some time ago, there was an operation of using kettle to realize data synchronization, including Installation and configuration of kettle, creation of job, creation of translate, etc. At that time, the time point of dead writing was used (that is, the data wil ...

Added by EODC on Sat, 29 Jan 2022 21:55:08 +0200

Spark sql learning notes -- DataFrame, Dataset and sql parsing principles

catalogue 1, SparkSession, DataFrame and Dataset 2, Spark Sql parsing 1. Overall overview 2. sql syntax parsing key objects 3, Spark LogicalPlan 1. Overall overview 2. LogicalPlan class structure system​ 3. Generated by analyzed logicalplan 1, SparkSession, DataFrame and Dataset 1. To use the sparksql function, you need to create a ...

Added by cute_girl on Fri, 24 Dec 2021 03:44:53 +0200