3.3 build ES environment
3.3.1 ES installation
The development environment uses the ES stand-alone environment to start the ES server.
Note: for the old ES environment, you can delete elasticsearch-
6.2.1\data\nodes directory to completely clean up the ES environment.
Install elasticsearch head and start it.
3.3.2 create index library
Create index library
Create XC course index library, one partition and 0 copies.
3.3.3 create mapping
Post http://localhost:9200/xc_course/doc/_mapping { " properties " : { " description " : { " analyzer " : " ik_max_word " , " search_analyzer " : " ik_smart " , " type " : " text " }, " grade " : { " type " : " keyword " }, " id " : { " type " : " keyword " }, " mt " : { " type " : " keyword " }, " name " : { " analyzer " : " ik_max_word " , " search_analyzer " : " ik_smart " , " type " : " text " }, " users " : { " index " : false, " type " : " text " }, " charge " : { " type " : " keyword " }, " valid " : { " type " : " keyword " }, " pic " : { " index " : false, " type " : " keyword " }, " qq " : { " index " : false, " type " : " keyword " }, " price " : { " type " : " float " }, " price_old " : { " type " : " float " }, " st " : { " type " : " keyword " }, " status " : { " type " : " keyword " }, " studymodel " : { " type " : " keyword " }, " teachmode " : { " type " : " keyword " }, " teachplan " : { " analyzer " : " ik_max_word " , " search_analyzer " : " ik_smart " , " type " : " text " }, " expires " : { " type " : " date " , " format " : " yyyy ‐ MM ‐ dd HH:mm:ss " }, " pub_time " : { " type " : " date " , " format " : " yyyy ‐ MM ‐ dd HH:mm:ss " }, " start_time " : { " type " : " date " , " format " : " yyyy ‐ MM ‐ dd HH:mm:ss " }, " end_time " : { " type " : " date " , " format " : " yyyy ‐ MM ‐ dd HH:mm:ss " } } }
3.4 Logstash creating index
Logstash is an open-source software under ES, which can collect data from multiple sources, transform data, and then send the data to the elastic search to create an index.
This project uses Logstash to apply data from MySQL to ES index.
3.4.1 download Logstash
Download logstash version 6.2.1, which is consistent with the elastic search version 6.2.1 used in this project.
Decompression:
3.4.2 install logstash input JDBC
Logstash input JDBC is developed by ruby. Download Ruby and install it first
Download address: https://rubyinstaller.org/downloads/
Download version 2.5.
Installation complete check if installation is successful
Logstash 5. X and above have their own logstash input JDBC. Version 6.x does not have its own logstash input JDBC plug-in, which needs to be installed manually
After the installation is successful, we can view the corresponding plug-in version in the following directory under the logstash root directory
3.4.3 create template file
Logstash's job is to read data from MySQL and create an index into ES. Here, it is necessary to create a mapping template file in advance for logstash to use.
Create XC ﹣ course ﹣ template.json in the confirmation directory of logstage, as follows:
The XC ﹣ course ﹣ template.json directory of this tutorial is: D: / elasticsearch / logstash-6.2.1/confirming/xc ﹣ course ﹣ template.json
{ " mappings " : { " doc " : { " properties " : { " charge " : { " type " : " keyword " }, " description " : { " analyzer " : " ik_max_word " , " search_analyzer " : " ik_smart " , " type " : " text " }, " end_time " : { " format " : " yyyy ‐ MM ‐ dd HH:mm:ss " , " type " : " date " }, " expires " : { " format " : " yyyy ‐ MM ‐ dd HH:mm:ss " , " type " : " date " }, " grade " : { " type " : " keyword " }, " id " : { " type " : " keyword " }, " mt " : { " type " : " keyword " }, " name " : { " analyzer " : " ik_max_word " , " search_analyzer " : " ik_smart " , " type " : " text " }, " pic " : { " index " : false, " type " : " keyword " }, " price " : { " type " : " float " }, " price_old " : { " type " : " float " }, " pub_time " : { " format " : " yyyy ‐ MM ‐ dd HH:mm:ss " , " type " : " date " }, " qq " : { " index " : false, " type " : " keyword " }, " st " : { " type " : " keyword " }, " start_time " : { " format " : " yyyy ‐ MM ‐ dd HH:mm:ss " , " type " : " date " }, " status " : { " type " : " keyword " }, " studymodel " : { " type " : " keyword " }, " teachmode " : { " type " : " keyword " }, " teachplan " : { " analyzer " : " ik_max_word " , " search_analyzer " : " ik_smart " , " type " : " text " }, " users " : { " index " : false, " type " : " text " }, " valid " : { " type " : " keyword " } } } }, " template " : " xc_course " }