es is developed based on java,
Use the new version of es
Minimum requirement jdk1 8，
Client, interface tools,
Official website: https://www.elastic.co/cn/
ES is document oriented and everything is json
Documents are pieces of data,
The type is the same as that of relational database (int, double,...)
An index, similar to a database, is a very large collection of documents.
Visual table display of inverted index:
If Chinese is used, ik word splitter is recommended.
Basic operation of index
1. Create an index
PUT / index name / type name / document id
{request body}
Type assignment:
Index creation rules: generally generated by default in java_ doc
GET information through GET request:
GET / index name (GET index information)
GET / index name / type name (GET type information)
GET / index name / type name / document id (GET the information of the document)
If the data type is not specified when adding data, es will configure the default type for us.
2. Modify the document
POST /Index name/Type name/file id/_update
{
"doc":{
(The specific document format is " xx":"xx")
}
}
3. Delete
DELETE / index name
DELETE / index name / type name
DELETE / index name / type name / document id
About document operations
PUT add data
GET query data
PUT updates data (it is recommended to use POST /.. / u update). POST has higher degrees of freedom
PUT is in the form of overwrite. All values need to be passed, including data that does not need to be updated. Otherwise, it will be overwritten and empty.
Simple search:
GET /. . /. . /. .
Simple condition query:
GET /. . /. . /. . /_ search? Q = XX: XX (generate basic query according to the default mapping rules)
Complex operation query: (sorting, paging, highlighting, fuzzy query, precise query) (build query)
GET /. . /. . /. . /_search
{
"query":{
"match":{
"xx":xx
}
}
}
There is an important parameter hits in the query result:
Query the specified field: use "_source"
GET /. . /. . /. . /_search
{
"query":{
"match":{
"xx":xx //query criteria
}
},
"_source":["xx",'xxx'] //Specified field, result filtering
}
Sorting and paging:
GET /. . /. . /. . /_search
{
"query":{
"match":{
"xx":xx
}
},
"sort":[{
"xx":{ //Sort by xx
"order":"desc" //In descending order, "asc" is in ascending order
}
}],
"from":0, //Starting from item 0 (pagination)
"size":10 //Display 10 data at a time
}
Boolean query: (multi condition query):
Must: equivalent to and in sql, which means that all conditions must be met.
short: equivalent to or, indicating that a condition is met
GET /. . /. . /. . /_search
{
"query":{
"bool":{
"must":[
{
"match":{
"xx":xx
}
},
{
"match":{
"xx":xx
}
}
]
}
}
}
Where, match can be changed to match_not means not:
filter:
gt: >
gte:>=
lt:<
lte:<=
Exact query:
The term tag is precisely searched by the term process specified by the inverted index.
About participle:
term is direct and accurate,
match will be parsed using a word splitter (first analyze the document, and then query through the analyzed document)
Highlight:
highlight label:
You can use pre_tags and post_tags custom highlight conditions:
Integrating springboot
1. Find native dependencies
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-high-level-client</artifactId>
<version>7.13.4</version>
</dependency>
2. Find object
initialization,
3. Guide Package:
Ensure that the imported dependency is consistent with the version of es.
<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <parent> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-parent</artifactId> <version>2.5.3</version> <relativePath/> <!-- lookup parent from repository --> </parent> <groupId>com.xct</groupId> <artifactId>es-user</artifactId> <version>0.0.1-SNAPSHOT</version> <name>es-user</name> <description>Demo project for Spring Boot</description> <properties> <java.version>1.8</java.version> </properties> <dependencies> <!--Import fastjson--> <dependency> <groupId>com.alibaba</groupId> <artifactId>fastjson</artifactId> <version>1.2.66</version> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-data-elasticsearch</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-devtools</artifactId> <scope>runtime</scope> <optional>true</optional> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-configuration-processor</artifactId> <optional>true</optional> </dependency> <dependency> <groupId>org.projectlombok</groupId> <artifactId>lombok</artifactId> <optional>true</optional> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope> </dependency> </dependencies> <build> <plugins> <plugin> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-maven-plugin</artifactId> <configuration> <excludes> <exclude> <groupId>org.projectlombok</groupId> <artifactId>lombok</artifactId> </exclude> </excludes> </configuration> </plugin> </plugins> </build> </project>
java implementation:
Create a config object and create a bean:
package com.xct.config;
import org.apache.http.HttpHost;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
@Configuration
public class ElasticSearchClientConfig {
@Bean
public RestHighLevelClient restHighLevelClient(){
RestHighLevelClient client = new RestHighLevelClient(
RestClient.builder(
new HttpHost("xxx.xxx.xxx.xxx",9200,"http")));
return client;
}
}
Create an ESconst with static constants for easy modification:
package com.xct.utils;
public class ESconst {
public static final String ES_INDEX = "test1";
}
Call the following test in the test class:
package com.xct;
import com.alibaba.fastjson.JSON;
import com.xct.pojo.DeviceProperties;
import com.xct.utils.ESconst;
import org.elasticsearch.action.admin.indices.delete.DeleteIndexRequest;
import org.elasticsearch.action.bulk.BulkRequest;
import org.elasticsearch.action.bulk.BulkResponse;
import org.elasticsearch.action.delete.DeleteRequest;
import org.elasticsearch.action.delete.DeleteResponse;
import org.elasticsearch.action.get.GetRequest;
import org.elasticsearch.action.get.GetResponse;
import org.elasticsearch.action.index.IndexRequest;
import org.elasticsearch.action.index.IndexResponse;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.action.support.master.AcknowledgedResponse;
import org.elasticsearch.action.update.UpdateRequest;
import org.elasticsearch.action.update.UpdateResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.client.indices.CreateIndexRequest;
import org.elasticsearch.client.indices.CreateIndexResponse;
import org.elasticsearch.client.indices.GetIndexRequest;
import org.elasticsearch.common.unit.TimeValue;
import org.elasticsearch.common.xcontent.XContentType;
import org.elasticsearch.index.query.MatchAllQueryBuilder;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.index.query.TermQueryBuilder;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.builder.SearchSourceBuilder;
import org.elasticsearch.search.fetch.subphase.FetchSourceContext;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import java.io.IOException;
import java.text.SimpleDateFormat;
import java.util.ArrayList;
import java.util.Date;
import java.util.concurrent.TimeUnit;
@SpringBootTest
class EsUserApplicationTests {
@Autowired
private RestHighLevelClient restHighLevelClient;
//Test index creation Request
@Test
void testCreateIndex() throws IOException {
//
CreateIndexRequest request = new CreateIndexRequest("test1");
CreateIndexResponse createIndexResponse =
restHighLevelClient.indices().create(request, RequestOptions.DEFAULT);
System.out.println(createIndexResponse);
}
@Test
//Test get index
void testExistIndex() throws IOException {
GetIndexRequest request = new GetIndexRequest("test1");
boolean exists = restHighLevelClient.indices().exists(request,RequestOptions.DEFAULT);
System.out.println(exists);
}
@Test
//Test delete index
void testDeleteIndex() throws IOException {
DeleteIndexRequest request = new DeleteIndexRequest("test1");
AcknowledgedResponse delete = restHighLevelClient.indices().delete(request,RequestOptions.DEFAULT);
System.out.println(delete.isAcknowledged());
}
@Test
//Test create document
void testAddDocument() throws IOException {
//create object
DeviceProperties deviceProperties = new DeviceProperties("temperature",20, new Date());
//Create request
IndexRequest request = new IndexRequest("test1");
//rule
request.id("1");
request.timeout(TimeValue.timeValueSeconds(1));
request.timeout("1s");
//Put our data into json
request.source(JSON.toJSONString(deviceProperties), XContentType.JSON);
//client poke request
IndexResponse indexResponse = restHighLevelClient.index(request,RequestOptions.DEFAULT);
System.out.println(indexResponse.toString());
System.out.println(indexResponse.status());
}
@Test
//The for loop is set to save every 5s
void testCreateProperties() throws IOException {
//Create request
IndexRequest request = new IndexRequest("test1");
for (int i = 0; i <100 ; i ++){
//create object
DeviceProperties deviceProperties = new DeviceProperties("temperature",5+i, new Date(new Date().getTime()+5000*i));
//rule
request.id(String.valueOf(i));
request.timeout(TimeValue.timeValueSeconds(1));
request.timeout("1s");
//Put our data into json
request.source(JSON.toJSONString(deviceProperties), XContentType.JSON);
//client poke request
IndexResponse indexResponse = restHighLevelClient.index(request,RequestOptions.DEFAULT);
}
}
@Test
//Get the document and judge whether it exists
void testIsExists() throws IOException {
GetRequest getRequest = new GetRequest("test1","1");
//Do not get returned_ The context of source
getRequest.fetchSourceContext(new FetchSourceContext(false));
getRequest.storedFields("_none_");
boolean exists = restHighLevelClient.exists(getRequest,RequestOptions.DEFAULT);
System.out.println(exists);
}
@Test
//Get information about the document
void testGetDocument() throws IOException {
GetRequest getRequest = new GetRequest("test1","1");
GetResponse getResponse = restHighLevelClient.get(getRequest,RequestOptions.DEFAULT);
System.out.println(getResponse.getSourceAsString());//Print document content
System.out.println(getResponse);//Everything returned is the same as the command
}
@Test
//Update document information
void testUpdateDocument() throws IOException {
UpdateRequest updateRequest = new UpdateRequest("test1","1");
updateRequest.timeout("1s");
DeviceProperties deviceProperties = new DeviceProperties("temperature",1,new Date());
updateRequest.doc(JSON.toJSONString(deviceProperties),XContentType.JSON);
UpdateResponse updateResponse = restHighLevelClient.update(updateRequest,RequestOptions.DEFAULT);
System.out.println(updateResponse.status());
}
@Test
//Delete document record
void testDeleteRequest() throws IOException {
DeleteRequest deleteRequest = new DeleteRequest("test1","1");
deleteRequest.timeout("1s");
DeleteResponse deleteResponse = restHighLevelClient.delete(deleteRequest,RequestOptions.DEFAULT);
System.out.println(deleteResponse.status());
}
@Test
//Mass insert
void testBulkRequest() throws IOException {
BulkRequest bulkRequest = new BulkRequest();
bulkRequest.timeout("10s");
ArrayList<DeviceProperties> deviceProperties = new ArrayList<>();
for (int i = 0; i<1000; i++){
deviceProperties.add(new DeviceProperties("temperature",0+i,new Date(new Date().getTime()+5000*i)));
}
for (int i = 0; i<deviceProperties.size(); i++){
//Batch insert
bulkRequest.add(
new IndexRequest("test1")
.id(""+(i+1))
.source(JSON.toJSONString(deviceProperties.get(i)),XContentType.JSON));
}
BulkResponse bulkResponse = restHighLevelClient.bulk(bulkRequest,RequestOptions.DEFAULT);
System.out.println(bulkResponse.hasFailures());//Whether it failed or not. If false is returned, it means success
}
@Test
//query
void testSearch() throws IOException {
SearchRequest searchRequest = new SearchRequest(ESconst.ES_INDEX);
//Build search criteria
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
//Query conditions can be implemented using the QueryBuilders tool class
//QueryBuilders.termQuery exact match
//QueryBuilders.matchAllQuery() matches all
TermQueryBuilder termQueryBuilder = QueryBuilders.termQuery("value",500);//Query data with value 500
MatchAllQueryBuilder matchAllQueryBuilder = QueryBuilders.matchAllQuery();
searchSourceBuilder.query(termQueryBuilder);//accurate
// searchSourceBuilder.query(matchAllQueryBuilder);// whole
searchSourceBuilder.timeout(new TimeValue(60, TimeUnit.SECONDS));
searchRequest.source(searchSourceBuilder);
SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);
System.out.println(JSON.toJSONString(searchResponse.getHits()));
System.out.println("==============================");
for (SearchHit documentFields : searchResponse.getHits().getHits()){
System.out.println(documentFields.getSourceAsMap());
}
}
}
ps: thanks for the teaching of crazy God