Optimized points (window functions) of sparksql over hivesql

Sometimes, a select statement contains multiple window functions whose window definitions (OVER clauses) may be the same or different. For the same windows, there is no need to partition and sort them again. We can merge them into a Window operator. such as The realization principle of window function in spark and hive Case in: select i ...

Added by serverman on Tue, 07 Apr 2020 17:52:21 +0300

ElasticSearch implements data paging (using bucket sort)

Be careful: es version at least 6.1 above Sentence: GET 76/sessions/_search { "size": 0, "query": { "bool": { "must": [ { "term": { "sid": { "value": "76e14832" } } }, { "range": { "v_ymd": { "format": "yyyy-MM-d ...

Added by ryanlwh on Mon, 06 Apr 2020 20:42:41 +0300

python operation excel import data to mysql

Requirement: the exchange code of excel is imported into mysql, with a total of 30W pieces of data. Navicat has its own functions. It can directly import excel data into mysql. There are too many databases and manual operation is very troublesome. A script is written with python. Use pip to import xlrd,pymysql library. ...

Added by uncleronin on Mon, 06 Apr 2020 19:22:11 +0300

Using SMO to programmatically generate SQL Server table data

As a part of ETL, sometimes you need to generate the Insert script of the data, and then copy the human flesh to another place for execution. Students who are familiar with SMSS all know that there is a task of generating scripts, which can generate database create scripts, and in fact can also produce data in tables. Automatic ETL can't even e ...

Added by michalchojno on Mon, 06 Apr 2020 17:33:04 +0300

Automated installation of unattended batch scripts for SQL Server 2014 Enterprise Edition

_Copyright Statement: This is an original article created by a blogger and may not be reproduced without the permission of the blogger. SQL Server series software is a relational database management system introduced by Microsoft.At a conference in San Francisco on April 16, 2014, Microsoft CEO Satia Nadela officially launched "SQL Ser ...

Added by blade_922 on Sun, 05 Apr 2020 08:45:43 +0300

JDBC of database

JDBC JDBC(Java Database Connectivity) Java database connection specification (a set of interfaces) provided by Sun company JDBC core class DriverManager create connection Connection connection class Statement execute sql statement ResultSet result set JDBC connection steps 1. Registration driver 2. ...

Added by thinguy on Sat, 04 Apr 2020 18:04:51 +0300

Executing SQL statements with PDO

There are two methods to execute SQL statements in PDO:    1. exec() is used to process statements that do not return SQL results, such as insert update delete create, etc., but it will return the number of affected rows If you are executing an insert statement, you can use the lastinsertid() method to get t ...

Added by flOid on Sat, 04 Apr 2020 11:29:40 +0300

mysqldump export data backup -- set gtid purged = off

When exporting, backing up and restoring Mysql database from the master database to the slave database, you need to pay attention to whether to enable GTID mode for the database. If you enable it, you should add the parameter -- set GTID purged = off to mysqldump data. See reprint for details: Here are the first 30 lines of a.sql exported with ...

Added by Jack_Slocum on Fri, 03 Apr 2020 08:48:03 +0300

Common scripts related to Oracle Performance

In the absence of visualization tools to monitor database performance, common scripts come in handy. Here are a few scripts related to Oracle Performance for your reference. The following scripts have passed the Oracle 10g test, and Oracle 11g may need to be adjusted accordingly. 1. Find SQL statements with the most buffer \ ...

Added by tarleton on Thu, 02 Apr 2020 18:31:04 +0300

The difference between sqlite query and rawQuery

query public Cursor query(boolean distinct, String table, String[] columns, String selection, String[] selectionArgs, String groupBy, String having, String orderBy, String limit, CancellationSignal cancellationSignal) //10 parameters //distinct: set to true, each row must have u ...

Added by ChrisDarl on Thu, 02 Apr 2020 09:37:46 +0300