ES getting started HTTP
Install ES
Download from ElasticSearch official website , take the latest version as an example, elasticsearch-7.17.0-windows-x86_64.zip
Unzip the compressed package, enter the bin directory in elasticsearch, and double-click elasticsearch Bat run, enter in the browser http://localhost:9200 , press enter, and the following message appears, indicating that elasticsearch is started successfully
Index operation (create, query, delete)
Create index
PUT: http://127.0.0.1:9200/shopping
Note: index creation cannot be requested by POST
Query index
1. Query the index of the specified name
GET: http://127.0.0.1:9200/shopping
2. Query all indexes
GET: http://127.0.0.1:9200/_cat/indices?v
Delete index
DELETE : http://127.0.0.1:9200/shopping
Document operation (create, query, modify, delete)
Create document/_ doc or/_ create
1. When id is specified, POST and PUT requests are equivalent, i.e. idempotent
POST: http://127.0.0.1:9200/shopping/_doc/1001
2. No id is specified, only POST request can be used
POST: http://127.0.0.1:9200/shopping/_doc
{
"title":"Mi phones",
"category":"millet",
"images":"http://www.gulixueyuan.com.xm.jpg",
"price":3999.0
}
Query document/_ search
1. Query the document with the specified id
GET: http://127.0.0.1:9200/shopping/_doc/1001
2. Query all documents under the index
GET: http://127.0.0.1:9200/shopping/_search
Modify document
1. Fully modify put & post (idempotent)/_ doc
POST: http://127.0.0.1:9200/shopping/_doc/1001
The parameter is the new json data
{
"title":"Huawei Mobile",
"category":"Huawei",
"images":"http://www.gulixueyuan.com.hw.jpg",
"price":5999.0
}
2. Partially modify post/_ update
POST: http://127.0.0.1:9200/shopping/_update/1001
Note: the_ doc is modified to_ update. _ doc means new, equivalent to_ create
Parameters also vary:
{
"doc":{
"title":"Meizu mobile phone"
}
}
remove document
DELETE: http://127.0.0.1:9200/shopping/_doc/1001
Advanced query (condition, paging, sorting)
1. ? Splicing query criteria
Syntax format:_ search?q=
Query Xiaomi brand mobile phone
GET: http://127.0.0.1:9200/shopping/_search?q=category: millet
2. body pass query criteria
POST: http://127.0.0.1:9200/shopping/_search
{
"query":{
"match":{
"category":"millet"
}
}
}
- Query represents query criteria
- match indicates a matching query
- match_all query all
{
"query":{
"match_all":{
}
}
}
3. Paging query
{
"query":{
"match_all":{
}
},
"from":0,
"size":3
}
- from: indicates the starting position of the current page data. Calculation formula: (page number - 1)*size
- size: indicates the number of data pieces per page
4. Filter and display query results
Only the title field is displayed in the query results
{
"query":{
"match_all":{
}
},
"from":0,
"size":3,
"_source":["title"]
}
- _ source: query result filtering, array format, indicating the fields to be displayed
5. Sorting of query results
Sort the query results in descending order of price
{
"query":{
"match_all":{
}
},
"from":0,
"size":3,
"_source":["title"],
"sort":{
"price":{
"order": "desc"
}
}
}
- Sort: indicates to sort the query results
- order: indicates the sorting method. asc: ascending, desc: descending
Multi condition query and range query
- bool: indicates multiple condition parameters
- must: array format, indicating that multiple conditions are met at the same time; should: indicates that at least one is satisfied
Query 2999 yuan Xiaomi mobile phone
{
"query":{
"bool":{
"must":[
{
"match":{
"category":"millet"
}
},
{
"match":{
"price":2999
}
}
]
}
}
}
Query Xiaomi or Huawei branded mobile phones
{
"query":{
"bool":{
"should":[
{
"match":{
"category":"millet"
}
},
{
"match":{
"category":"Huawei"
}
}
]
}
}
}
Query Xiaomi or Huawei mobile phones with prices in the range of 2000-3000
{
"query":{
"bool":{
"should":[
{
"match":{
"category":"millet"
}
},
{
"match":{
"category":"Huawei"
}
}
],
"filter":{
"range":{
"price":{
"gt" : 2000,
"lt" : 3000
}
}
}
}
}
}
- filter: indicates filtering
- Range: indicates the range
- gt: indicates greater than
- gte: indicates greater than or equal to
- lt: less than
Full text retrieval, highlighting
ES performs word segmentation when storing or querying. When match is used for matching, the query result is the query result after word segmentation. For example:
{
"query":{
"match":{
"category":"Xiaohua"
}
}
}
The query result of the above code contains all the data of "small" or "Hua", and the query conditions are segmented.
If you don't want to segment the query criteria, use match_phrase
{
"query":{
"match_phrase":{
"category":"Xiaohua"
}
}
}
- match_phrase: indicates that word segmentation is not performed for query criteria
Search results highlight
Query Xiaomi brand mobile phones and highlight them
{
"query":{
"match_phrase":{
"category":"millet"
}
},
"highlight":{
"fields":{
"category":{}
}
}
}
- highlight: highlight
- Fields: highlighted fields
Aggregation operation
1. Grammar
"aggregations" : {
"<aggregation_name>" : { <!--Aggregate name -->
"<aggregation_type>" : { <!--Type of aggregation -->
<aggregation_body> <!--Aggregate: which fields are aggregated -->
}
[,"meta" : { [<meta_data_body>] } ]? <!--element -->
[,"aggregations" : { [<sub_aggregation>]+ } ]? <!--In aggregation, sub aggregation is defined -->
}
}
2. Type of aggregation
- sum: sum
- max: maximum
- min: minimum
- avg: average value
- terms: group and count
- cardinality: de duplication count
- stats: statistics, which will directly display various aggregation results, including total records, maximum value, minimum value, average value and summary (summation) value
- percentiles: percentage. For the value of the specified field, accumulate the proportion of the number of documents corresponding to each value from small to large, and return the value corresponding to the specified proportion
3. Examples
Group mobile phones with different prices
{
"aggs":{ // Aggregation operation
"price_group":{ // Name at will
"terms":{ // grouping
"field":"price" // Grouping field
}
}
},
"size":0
}
- "size": 0, which means no original data is attached and only aggregation results are displayed
Find the average of all mobile phone prices
{
"aggs":{ // Aggregation operation
"price_avg":{ // Name at will
"avg":{ // average value
"field":"price" // Average field
}
}
},
"size":0
}
Price statistics
{
"aggs":{
"price_stats":{
"stats":{
"field":"price"
}
}
},
"size":0
}
Multi value query - maximum, minimum and average
{
"aggs":{
"price_max":{
"max":{
"field":"price"
}
},
"price_min":{
"min":{
"field":"price"
}
},
"price_avg":{
"avg":{
"field":"price"
}
}
},
"size":0
}
Aggregate after grouping, and calculate the average salary according to gender grouping
{
"aggs":{
"category":{
"terms":{
"field":"sex"
},
"aggs":{
"salary_avg":{
"avg":{
"field":"salary"
}
}
}
}
},
"size":0
}
Note: ES cannot aggregate string type fields
Mapping relationship_ mapping
Create user index
GET: http://127.0.0.1:9200/user
Add mapping relationship to user
PUT: http://127.0.0.1:9200/user/_mapping
{
"properties" : { //Define attributes
"name" : {
"type" : "text", //The field type text indicates separable words
"index" : true //Can a field be indexed
},
"sex" : {
"type" : "keyword", //The field type keyword represents a non separable word
"index" : true
},
"phone" : {
"type" : "keyword", //The field type keyword represents a non separable word
"index" : false
}
}
}
Query the mapping relationship of user
GET: http://127.0.0.1:9200/user/_mapping
After defining the mapping relationship, you can perform match retrieval for user. The specific retrieval method is omitted here.
ES getting started Java API
Environmental preparation
maven dependency
<!--elasticsearch-->
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
<version>7.8.0</version>
</dependency>
<!--elasticsearch Advanced client-->
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-high-level-client</artifactId>
<version>7.8.0</version>
<!--Solve the problem of creating index errors caused by inconsistent client versions-->
<exclusions>
<exclusion>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-client</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-client</artifactId>
<version>7.8.0</version>
</dependency>
<!--jackson-databind-->
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>2.9.9</version>
</dependency>
Create es client
public class ESDemo_Client {
public static void main(String[] args) throws IOException {
// 1. Create ES client
RestHighLevelClient esClient = new RestHighLevelClient(
RestClient.builder(new HttpHost("localhost",9200,"http"))
);
// Close ES client
esClient.close();
}
}
Index operation (create, query, delete)
Create index
// 2. Index creation
CreateIndexRequest request = new CreateIndexRequest("user");
CreateIndexResponse createIndexResponse =
esClient.indices().create(request, RequestOptions.DEFAULT);
// Response status
boolean acknowledged = createIndexResponse.isAcknowledged();
System.out.println("Create index:" + acknowledged);
Index query
// 3. Index operation query index
GetIndexRequest request = new GetIndexRequest("user");
GetIndexResponse getIndexResponse =
esClient.indices().get(request, RequestOptions.DEFAULT);
// Response information
System.out.println("getAliases() = " + getIndexResponse.getAliases());
System.out.println("getMappings() = " + getIndexResponse.getMappings());
System.out.println("getSettings() = " + getIndexResponse.getSettings());
Delete index
// 4. Index operation delete index
DeleteIndexRequest request = new DeleteIndexRequest("user");
AcknowledgedResponse deleteResponse =
esClient.indices().delete(request, RequestOptions.DEFAULT);
// Response status
boolean acknowledged = deleteResponse.isAcknowledged();
System.out.println("Delete index:" + acknowledged);
Document operation (create, modify, query, delete)
create documents
ic class ESDemo_Doc {
public static void main(String[] args) throws IOException {
// 1. Create ES client
RestHighLevelClient esClient = new RestHighLevelClient(
RestClient.builder(new HttpHost("localhost", 9200, "http"))
);
// 2. Document operation create document
IndexRequest request = new IndexRequest();
// Specify index and id
request.index("user").id("1001");
// Prepare data
User user = new User();
user.setName("zhangsan");
user.setSex("male");
user.setAge(30);
// To insert data into ES, the data must be converted to JSON format
ObjectMapper mapper = new ObjectMapper();
String userJson = mapper.writeValueAsString(user);
request.source(userJson, XContentType.JSON);
// create documents
IndexResponse response = esClient.index(request, RequestOptions.DEFAULT);
System.out.println("response = " + response.getResult());
// Close ES client
esClient.close();
}
Modify document
// 3. Document operation modify document
UpdateRequest request = new UpdateRequest();
// Specify index and id
request.index("user").id("1001");
// Modify data
request.doc(XContentType.JSON, "sex", "female");
// Modify document
UpdateResponse response = esClient.update(request, RequestOptions.DEFAULT);
System.out.println("response = " + response.getResult());
consult your documentation
// 4. Query data
GetRequest request = new GetRequest();
// Specify the index and id of the query
request.index("user").id("1001");
GetResponse response = esClient.get(request, RequestOptions.DEFAULT);
// Result output
System.out.println(response.getSourceAsString());
remove document
// 5. Delete document
DeleteRequest request = new DeleteRequest();
// Specify the index and id to delete
request.index("user").id("1001");
DeleteResponse response = esClient.delete(request, RequestOptions.DEFAULT);
System.out.println(response.getResult());
Document operation (batch add, batch delete)
Batch add
public class ESDemo_Doc_Batch {
public static void main(String[] args) throws IOException {
// 1. Create ES client
RestHighLevelClient esClient = new RestHighLevelClient(
RestClient.builder(new HttpHost("localhost", 9200, "http"))
);
// 2. Batch adding of document operation
BulkRequest request = new BulkRequest();
// Prepare data
request.add(new IndexRequest("user").id("1001").source(XContentType.JSON,"name","zhangsan"));
request.add(new IndexRequest("user").id("1002").source(XContentType.JSON,"name","lisi"));
request.add(new IndexRequest("user").id("1003").source(XContentType.JSON,"name","wangwu"));
// Batch add
BulkResponse response = esClient.bulk(request, RequestOptions.DEFAULT);
System.out.println("Time consuming:" + response.getTook());
System.out.println(Arrays.toString(response.getItems()));
// Close ES client
esClient.close();
}
}
Batch delete
// 3. Batch deletion of document operation
BulkRequest request = new BulkRequest();
// Prepare data
request.add(new DeleteRequest("user").id("1001"));
request.add(new DeleteRequest("user").id("1002"));
request.add(new DeleteRequest("user").id("1003"));
// Batch delete
BulkResponse response = esClient.bulk(request, RequestOptions.DEFAULT);
System.out.println("Time consuming:" + response.getTook());
System.out.println(Arrays.toString(response.getItems()));
Document operation - advanced query (full quantity, condition, paging, sorting, field filtering)
- SearchRequest: query request object
- SearchSourceBuilder: Query Builder
- QueryBuilder: condition constructor
1. Query all data querybuilders matchAllQuery()
// 1. Query all data in the index
SearchRequest request = new SearchRequest();
request.indices("user");
// Prepare query criteria
request.source(new SearchSourceBuilder().query(QueryBuilders.matchAllQuery()));
// query
SearchResponse searchResponse = esClient.search(request, RequestOptions.DEFAULT);
System.out.println("Query time: " + searchResponse.getTook());
// Query result traversal
SearchHits hits = searchResponse.getHits();
System.out.println("Total records: " + hits.getTotalHits());
for (SearchHit hit : hits) {
System.out.println(hit.getSourceAsString());
}
2. Query criteria: querybuilders termQuery()
// 2. Condition query
SearchRequest request = new SearchRequest();
request.indices("user");
// Prepare query criteria
request.source(new SearchSourceBuilder().query(QueryBuilders.termQuery("age",30)));
// query
SearchResponse searchResponse = esClient.search(request, RequestOptions.DEFAULT);
System.out.println("Query time: " + searchResponse.getTook());
// Query result traversal
SearchHits hits = searchResponse.getHits();
System.out.println("Total records: " + hits.getTotalHits());
for (SearchHit hit : hits) {
System.out.println(hit.getSourceAsString());
}
3. Paging query searchsourcebuilder from() searchSourceBuilder. size()
// 3. Paging query
SearchRequest request = new SearchRequest();
request.indices("user");
// Prepare paging information
SearchSourceBuilder searchBuilder = new SearchSourceBuilder().query(QueryBuilders.matchAllQuery());
searchBuilder.from(0); //The starting position of the current page, (page number - 1)*size
searchBuilder.size(2); //Number of entries per page
request.source(searchBuilder);
// query
SearchResponse searchResponse = esClient.search(request, RequestOptions.DEFAULT);
System.out.println("Query time: " + searchResponse.getTook());
// Query result traversal
SearchHits hits = searchResponse.getHits();
System.out.println("Total records: " + hits.getTotalHits());
for (SearchHit hit : hits) {
System.out.println(hit.getSourceAsString());
}
4. Sort query results searchsourcebuilder sort()
// 4. Sorting of query results
SearchRequest request = new SearchRequest();
request.indices("user");
// Query all data
SearchSourceBuilder searchBuilder = new SearchSourceBuilder().query(QueryBuilders.matchAllQuery());
// The query results are arranged in descending order by age
searchBuilder.sort("age", SortOrder.DESC);
request.source(searchBuilder);
// query
SearchResponse searchResponse = esClient.search(request, RequestOptions.DEFAULT);
System.out.println("Query time: " + searchResponse.getTook());
// Query result traversal
SearchHits hits = searchResponse.getHits();
System.out.println("Total records: " + hits.getTotalHits());
for (SearchHit hit : hits) {
System.out.println(hit.getSourceAsString());
}
5. Field filtering searchsourcebuilder fetchSource()
// 5. Field filtering
SearchRequest request = new SearchRequest();
request.indices("user");
// Query all data
SearchSourceBuilder searchBuilder = new SearchSourceBuilder().query(QueryBuilders.matchAllQuery());
// Field filtering
String[] includes = {"name"}; //Required fields
String[] excludes = {}; //Excluded fields
searchBuilder.fetchSource(includes, excludes);
request.source(searchBuilder);
// query
SearchResponse searchResponse = esClient.search(request, RequestOptions.DEFAULT);
System.out.println("Query time: " + searchResponse.getTook());
// Query result traversal
SearchHits hits = searchResponse.getHits();
System.out.println("Total records: " + hits.getTotalHits());
for (SearchHit hit : hits) {
System.out.println(hit.getSourceAsString());
}
Document operation - advanced query (multi condition query, range query)
6. Combined query querybuilders boolQuery()
// 6. Combined query
SearchRequest request = new SearchRequest();
request.indices("user");
SearchSourceBuilder searchBuilder = new SearchSourceBuilder();
// Query users aged 30 and female
BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();
boolQueryBuilder.must(QueryBuilders.matchQuery("age", 30));
boolQueryBuilder.must(QueryBuilders.matchQuery("sex", "female"));
searchBuilder.query(boolQueryBuilder);
request.source(searchBuilder);
// query
SearchResponse searchResponse = esClient.search(request, RequestOptions.DEFAULT);
System.out.println("Query time: " + searchResponse.getTook());
// Query result traversal
SearchHits hits = searchResponse.getHits();
System.out.println("Total records: " + hits.getTotalHits());
for (SearchHit hit : hits) {
System.out.println(hit.getSourceAsString());
}
7. Range query querybuilders rangeQuery()
// 7. Range query
SearchRequest request = new SearchRequest();
request.indices("user");
SearchSourceBuilder searchBuilder = new SearchSourceBuilder();
// Query users aged 30 ~ 40
RangeQueryBuilder rangeQuery = QueryBuilders.rangeQuery("age");
rangeQuery.gte(30); //30 or more
rangeQuery.lte(40); //Up to 40
searchBuilder.query(rangeQuery);
request.source(searchBuilder);
// query
SearchResponse searchResponse = esClient.search(request, RequestOptions.DEFAULT);
System.out.println("Query time: " + searchResponse.getTook());
// Query result traversal
SearchHits hits = searchResponse.getHits();
System.out.println("Total records: " + hits.getTotalHits());
for (SearchHit hit : hits) {
System.out.println(hit.getSourceAsString());
}
Document operation - advanced query (full-text retrieval, highlighting)
8. Full text retrieval (fuzzy query) querybuilders fuzzyQuery()
// 8. Full text search (fuzzy query)
SearchRequest request = new SearchRequest();
request.indices("user");
SearchSourceBuilder searchBuilder = new SearchSourceBuilder();
// Specify the content of the fuzzy query and the deviation distance
FuzzyQueryBuilder fuzzyQuery =
QueryBuilders.fuzzyQuery("name", "wangwu").fuzziness(Fuzziness.TWO);
searchBuilder.query(fuzzyQuery);
request.source(searchBuilder);
// query
SearchResponse searchResponse = esClient.search(request, RequestOptions.DEFAULT);
System.out.println("Query time: " + searchResponse.getTook());
// Query result traversal
SearchHits hits = searchResponse.getHits();
System.out.println("Total records: " + hits.getTotalHits());
for (SearchHit hit : hits) {
System.out.println(hit.getSourceAsString());
}
9. Highlight HighlightBuilder in the query result
// 9. Highlight query results
SearchRequest request = new SearchRequest();
request.indices("user");
SearchSourceBuilder searchBuilder = new SearchSourceBuilder();
// Query the user named Wang Wu. Note that when querying Chinese, you need to use attributes keyword
TermQueryBuilder termQueryBuilder = QueryBuilders.termQuery("name.keyword", "Wang Wu");
// Highlight settings
HighlightBuilder highlightBuilder = new HighlightBuilder();
highlightBuilder.field("name"); //Highlight field
highlightBuilder.preTags("<font color='red'>"); //Prefix label
highlightBuilder.postTags("</font>"); //Suffix label
searchBuilder.highlighter(highlightBuilder);
searchBuilder.query(termQueryBuilder);
request.source(searchBuilder);
// query
SearchResponse searchResponse = esClient.search(request, RequestOptions.DEFAULT);
System.out.println("Query time: " + searchResponse.getTook());
// Query result traversal
SearchHits hits = searchResponse.getHits();
System.out.println("Total records: " + hits.getTotalHits());
for (SearchHit hit : hits) {
System.out.println(hit.getSourceAsString());
}
Note: when termQuery() queries Chinese, you need to use attributes keyword
QueryBuilders.termQuery("name.keyword", "Wang Wu");
Document operation - advanced query (aggregate operation)
1. The maximum value is aggregationbuilders max()
// 1. Aggregation operation - Maximum
SearchRequest request = new SearchRequest();
request.indices("user");
SearchSourceBuilder searchBuilder = new SearchSourceBuilder();
// Maximum age aggregationbuilders max
AggregationBuilder aggregationBuilder =
AggregationBuilders.max("maxAge").field("age");
searchBuilder.aggregation(aggregationBuilder);
request.source(searchBuilder);
// query
SearchResponse searchResponse = esClient.search(request, RequestOptions.DEFAULT);
System.out.println("Query time: " + searchResponse.getTook());
// Query result traversal
SearchHits hits = searchResponse.getHits();
System.out.println("Total records: " + hits.getTotalHits());
for (SearchHit hit : hits) {
System.out.println(hit.getSourceAsString());
}
2. Group count aggregationbuilders terms()
// 11. Aggregation operation - group count
SearchRequest request = new SearchRequest();
request.indices("user");
SearchSourceBuilder searchBuilder = new SearchSourceBuilder();
// Group count by age
AggregationBuilder aggregationBuilder =
AggregationBuilders.terms("ageGroups").field("age");
searchBuilder.aggregation(aggregationBuilder);
request.source(searchBuilder);
// query
SearchResponse searchResponse = esClient.search(request, RequestOptions.DEFAULT);
System.out.println("Query time: " + searchResponse.getTook());
// Query result traversal
SearchHits hits = searchResponse.getHits();
System.out.println("Total records: " + hits.getTotalHits());
for (SearchHit hit : hits) {
System.out.println(hit.getSourceAsString());
}