Writing requirements written to the search module in the previous period of time, with more and more conditions, more and more complex combination of various conditions, and poor maintenance in the later period, so improved the previous search writing, wrote a small case in spare time, and discussed with you.
Complete case
Based on es7.14, using ik_smart participle
Use olivere/elastic go-es extension
Use gin routing
Batch Generation test data
Batch Import Data to es case
v1 code before improvement & V2 code after improvement
Request Structure
SearchRequest Search Request Structure
// SearchRequest Search Request Structure
type SearchRequest struct {
Keyword string `form:"keyword"` // Key word
CategoryId uint8 `form:"category_id"` // classification
Sort uint8 `form:"sort" binding:"omitempty,oneof=1 2 3"` // Sort 1 = Views; 2 = Collections; 3 = Comments;
IsSolve uint8 `form:"is_solve" binding:"omitempty,oneof=1 2"` // Is it resolved
Page int `form:"page,default=1"` // The number of pages
PageSize int `form:"page_size,default=10"` // Transactions per page
}Before improvement
Search() article search is almost the same code as Recommend() article recommendation, except that the conditions are different, there is too much duplicate code, and it is not easy to maintain.
Example: Search() processes search requests
func (a ArticleV1) Search(c *gin.Context) {
req := new(model.SearchRequest)
if err := c.ShouldBind(req); err != nil {
c.JSON(400, err.Error())
return
}
// Build Search
builder := es.Client.Search().Index(model.ArticleEsAlias)
bq := elastic.NewBoolQuery()
// Title
if req.Keyword != "" {
builder.Query(bq.Must(elastic.NewMatchQuery("title", req.Keyword)))
}
// classification
if req.CategoryId != 0 {
builder.Query(bq.Filter(elastic.NewTermQuery("category_id", req.CategoryId)))
}
// Is it resolved
if req.IsSolve != 0 {
builder.Query(bq.Filter(elastic.NewTermQuery("is_solve", req.IsSolve)))
}
// sort
switch req.Sort {
case SortBrowseDesc:
builder.Sort("brows_num", false)
case SortUpvoteDesc:
builder.Sort("upvote_num", false)
case SortCollectDesc:
builder.Sort("collect_num", false)
default:
builder.Sort("created_at", false)
}
// paging
from := (req.Page - 1) * req.PageSize
// Specify Query Fields
include := []string{"id", "category_id", "title", "brows_num", "collect_num", "upvote_num", "is_recommend", "is_solve", "created_at"}
builder.
FetchSourceContext(
elastic.NewFetchSourceContext(true).Include(include...),
).
From(from).
Size(req.PageSize)
// Execute Query
do, err := builder.Do(context.Background())
if err != nil {
c.JSON(500, err.Error())
return
}
// Get the number matched
total := do.TotalHits()
// Serialized data
list := make([]model.SearchResponse, len(do.Hits.Hits))
for i, raw := range do.Hits.Hits {
tmpArticle := model.SearchResponse{}
if err := json.Unmarshal(raw.Source, &tmpArticle); err != nil {
log.Println(err)
}
list[i] = tmpArticle
}
c.JSON(http.StatusOK, gin.H{
"code": 200,
"data": gin.H{
"total": total,
"list": list,
},
})
return
}Request a test
curl GET '127.0.0.1:8080/article/v1/search?keyword=Fennel beans&page=1&page_size=2&sort=1'
Click here to see the results returned
{
"code": 200,
"data": {
"list": [
{
"id": 8912,
"category_id": 238,
"title": "Fennel beans on the bill; funny,",
"brows_num": 253,
"collect_num": 34,
"upvote_num": 203,
"is_recommend": 2,
"is_solve": 1,
"created_at": "2021-02-01 15:19:36"
}
...
],
"total": 157
}
}
Recommend() Processing recommendation requests
func (a ArticleV1) Recommend(c *gin.Context) {}
// Recommend Article Recommendation
func (a ArticleV1) Recommend(c *gin.Context) {
// Build Search
builder := es.Client.Search().Index(model.ArticleEsAlias)
bq := elastic.NewBoolQuery()
builder.Query(bq.Filter(
// Recommended Articles
elastic.NewTermQuery("category_id", model.ArticleIsRecommendYes),
// resolved
elastic.NewTermQuery("is_solve", model.ArticleIsSolveYes),
))
// Browse sorting
builder.Sort("brows_num", false)
do, err := builder.From(0).Size(10).Do(context.Background())
if err != nil {
return
}
// Serialized data
...
Improved
Look first at the results
It is convenient to separate all the conditions of a query and query es like a database. You only need to combine the required conditions to get the desired results.
// Search Article Search
func (a ArticleV2) Search(c *gin.Context) {
req := new(model.SearchRequest)
if err := c.ShouldBind(req); err != nil {
c.JSON(400, err.Error())
return
}
// Query by adding conditions as easily as by looking up a database
list, total, err := service.NewArticle().
WhereKeyword(req.Keyword).
WhereCategoryId(req.CategoryId).
WhereIsSolve(req.IsSolve).
Sort(req.Sort).
Paginate(req.Page, req.PageSize).
DecodeSearch()
if err != nil {
c.JSON(400, err.Error())
return
}
c.JSON(http.StatusOK, gin.H{
"code": 200,
"data": gin.H{
"total": total,
"list": list,
},
})
return
}
// Recommend Article Recommendation
func (a ArticleV2) Recommend(c *gin.Context) {
// Query by adding conditions as easily as by looking up a database
list, _, err := service.NewArticle().
WhereCategoryId(model.ArticleIsRecommendYes).
WhereIsSolve(model.ArticleIsSolveYes).
OrderByDesc("brows_num").
PageSize(5).
DecodeRecommend()
if err != nil {
c.JSON(400, err.Error())
return
}
c.JSON(http.StatusOK, gin.H{
"code": 200,
"data": gin.H{
"total": len(list),
"list": list,
},
})
return
}
How do I do that?
Since only the conditions are different, assemble the required conditions in a combined manner before executing the query
internal/service/article.go
In this case, the must filter sort condition of es is involved, so we'll first construct a struct to hold the condition to be combined
type article struct {
must []elastic.Query
filter []elastic.Query
sort []elastic.Sorter
from int
size int
}Constructor
func NewArticle() *article {
return &article{
must: make([]elastic.Query, 0),
filter: make([]elastic.Query, 0),
sort: make([]elastic.Sorter, 0),
from: 0,
size: 10,
}
}Method of adding condition of combination
I used the AddKeyword() WithKeyword method name at first, but I didn't feel very good about it. Then I came to think of a quick way to query criteria in Laravel, such as where Id(1), which generates sql:xxx where id = 1, so I named it like Laravel did.
// WhereKeyword keywords
func (a article) WhereKeyword(keyword string) article {
if keyword != "" {
a.must = append(a.must, elastic.NewMatchQuery("title", keyword))
}
return a
}
// WhereCategoryId Classification
func (a article) WhereCategoryId(categoryId uint8) article {
if categoryId != 0 {
a.filter = append(a.filter, elastic.NewTermQuery("category_id", categoryId))
}
return a
}
// WhereIsSolve Resolved
func (a article) WhereIsSolve(isSolve uint8) article {
if isSolve != 0 {
a.filter = append(a.filter, elastic.NewTermQuery("is_solve", isSolve))
}
return a
}
// Sort Sorting
func (a article) Sort(sort uint8) article {
switch sort {
case SortBrowseDesc:
return a.OrderByDesc("brows_num")
case SortUpvoteDesc:
return a.OrderByDesc("upvote_num")
case SortCollectDesc:
return a.OrderByDesc("collect_num")
}
return a
}
// OrderByDesc Sorts Reversely by Field
func (a article) OrderByDesc(field string) article {
a.sort = append(a.sort, elastic.SortInfo{Field: field, Ascending: false})
return a
}
// OrderByAsc Sorts Positive Order by Field
func (a article) OrderByAsc(field string) article {
a.sort = append(a.sort, elastic.SortInfo{Field: field, Ascending: true})
return a
}
// Paginate Paging
// Page current page number
// Number of pageSize per page
func (a article) Paginate(page, pageSize int) article {
a.from = (page - 1) * pageSize
a.size = pageSize
return a
}
Now that you've built all the conditions you need, now that you have the conditions, you need to perform the final search
// Searcher Execute Query
func (a article) Searcher(include ...interface{}) ([]json.RawMessage, int64, error) {
builder := es.Client.Search().Index(model.ArticleEsAlias)
// Field of Query
includeKeys := make([]string, 0)
if len(include) > 0 {
includeKeys = structer.Keys(include[0])
}
// Build Query
builder.Query(
// Build bool query condition
elastic.NewBoolQuery().Must(a.must...).Filter(a.filter...),
)
// Execute Query
do, err := builder.
FetchSourceContext(elastic.NewFetchSourceContext(true).Include(includeKeys...)).
From(a.from).
Size(a.size).
SortBy(a.sort...).
Do(context.Background())
if err != nil {
return nil, 0, err
}
total := do.TotalHits()
list := make([]json.RawMessage, len(do.Hits.Hits))
for i, hit := range do.Hits.Hits {
list[i] = hit.Source
}
return list, total, nil
}Here you can get results by combining different conditions for different interface conditions using the pattern of the combination conditions built above
list, _, err := service.NewArticle().
WhereCategoryId(model.ArticleIsRecommendYes).
WhereIsSolve(model.ArticleIsSolveYes).
OrderByDesc("brows_num").
PageSize(5).
Searcher()But there's still a problem at this point, where the list returns an array of []json.RawMessage, and each record is a JSON string. Sometimes we need to further process the queried data, at this point we need to serialize the content into a corresponding struct, and we can write a DecodeXxx()Method to serialize the corresponding struct for different purposes
Recommend the structure of the result so that it returns a structure that is available for []model.RecommendResponse, instead of calling Searcher() directly and DecodeRecommend() when called.
func (a article) DecodeRecommend() ([]model.RecommendResponse, int64, error) {
rawList, total, err := a.Searcher(new(model.RecommendResponse))
if err != nil {
return nil, total, err
}
list := make([]model.RecommendResponse, len(rawList))
for i, raw := range rawList {
tmp := model.RecommendResponse{}
if err := json.Unmarshal(raw, &tmp); err != nil {
log.Println(err)
continue
}
list[i] = tmp
}
return list, total, nil
}summary
Now that you've finished the simple encapsulation of your search, the complete case is ready to go github.com/zxr615/go-escase Communications