• brief introduction

• query

  • Query process
  • Build query SQL statement
  • Conditional statement
  • Summary

• search structure

  • Definition of search
  • search method
  • Summary
• summary

brief introduction

GORM source code interpretation, based on v1.9.11 edition.

query

In the previous section, we explored how models are defined and how data tables are created
This time, take a look at how the query is implemented

Queries involve a large amount of content because they support various types of methods
Take a look at the simplest query methods available in the official documents

// Query the first record according to the primary key
db.First(&user)
//// SELECT * FROM users ORDER BY id LIMIT 1;

// Get a record at random
db.Take(&user)
//// SELECT * FROM users LIMIT 1;

// Query the last record according to the primary key
db.Last(&user)
//// SELECT * FROM users ORDER BY id DESC LIMIT 1;

// Query all records
db.Find(&users)
//// SELECT * FROM users;

// Query a specified record (only available if the primary key is integer)
db.First(&user, 10)
//// SELECT * FROM users WHERE id = 10;

Take the First method as an example to see its implementation:

// First find first record that match given conditions, order by primary key
func (s *DB) First(out interface{}, where ...interface{}) *DB {
    newScope := s.NewScope(out)
    newScope.Search.Limit(1)

    return newScope.Set("gorm:order_by_primary_key", "ASC").
        inlineCondition(where...).callCallbacks(s.parent.callbacks.queries).db
}

The First method obtains the First data from the database and sorts it in ascending primary key order

As mentioned earlier, the specific implementation of database operation depends on callbacks callbacks.queries .

In the default callbacks, three different query callback functions are registered

// Define callbacks for querying
func init() {
    DefaultCallback.Query().Register("gorm:query", queryCallback)
    DefaultCallback.Query().Register("gorm:preload", preloadCallback)
    DefaultCallback.Query().Register("gorm:after_query", afterQueryCallback)
}

Query process

Let's first look at the main queryCallback function

// queryCallback used to query data from database
func queryCallback(scope *Scope) {
    if _, skip := scope.InstanceGet("gorm:skip_query_callback"); skip {
        return
    }

    //we are only preloading relations, dont touch base model
    if _, skip := scope.InstanceGet("gorm:only_preload"); skip {
        return
    }

    defer scope.trace(scope.db.nowFunc())

    var (
        isSlice, isPtr bool
        resultType     reflect.Type
        results        = scope.IndirectValue()
    )

    if orderBy, ok := scope.Get("gorm:order_by_primary_key"); ok {
        if primaryField := scope.PrimaryField(); primaryField != nil {
            scope.Search.Order(fmt.Sprintf("%v.%v %v", scope.QuotedTableName(), scope.Quote(primaryField.DBName), orderBy))
        }
    }

    if value, ok := scope.Get("gorm:query_destination"); ok {
        results = indirect(reflect.ValueOf(value))
    }

    if kind := results.Kind(); kind == reflect.Slice {
        isSlice = true
        resultType = results.Type().Elem()
        results.Set(reflect.MakeSlice(results.Type(), 0, 0))

        if resultType.Kind() == reflect.Ptr {
            isPtr = true
            resultType = resultType.Elem()
        }
    } else if kind != reflect.Struct {
        scope.Err(errors.New("unsupported destination, should be slice or struct"))
        return
    }

    scope.prepareQuerySQL()

    if !scope.HasError() {
        scope.db.RowsAffected = 0
        if str, ok := scope.Get("gorm:query_option"); ok {
            scope.SQL += addExtraSpaceIfExist(fmt.Sprint(str))
        }

        if rows, err := scope.SQLDB().Query(scope.SQL, scope.SQLVars...); scope.Err(err) == nil {
            defer rows.Close()

            columns, _ := rows.Columns()
            for rows.Next() {
                scope.db.RowsAffected++

                elem := results
                if isSlice {
                    elem = reflect.New(resultType).Elem()
                }

                scope.scan(rows, columns, scope.New(elem.Addr().Interface()).Fields())

                if isSlice {
                    if isPtr {
                        results.Set(reflect.Append(results, elem.Addr()))
                    } else {
                        results.Set(reflect.Append(results, elem))
                    }
                }
            }

            if err := rows.Err(); err != nil {
                scope.Err(err)
            } else if scope.db.RowsAffected == 0 && !isSlice {
                scope.Err(ErrRecordNotFound)
            }
        }
    }
}

The core steps are scope.prepareQuerySQL() build SQL statement
Then through rows, err:= scope.SQLDB ().Query( scope.SQL , scope.SQLVars ...), database query executed

So how and to whom are the query results delivered?

Results are defined at the beginning of the function= scope.IndirectValue (), this is where the final query results belong

results can only be a structure or a slice of a structure

if kind := results.Kind(); kind == reflect.Slice {
  isSlice = true
  resultType = results.Type().Elem()
  results.Set(reflect.MakeSlice(results.Type(), 0, 0))

  if resultType.Kind() == reflect.Ptr {
    isPtr = true
    resultType = resultType.Elem()
  }
} else if kind != reflect.Struct {
  scope.Err(errors.New("unsupported destination, should be slice or struct"))
  return
}

How to handle the query results is shown in the following code:

columns, _ := rows.Columns()
for rows.Next() {
  scope.db.RowsAffected++

  elem := results
  if isSlice {
    elem = reflect.New(resultType).Elem()
  }

  scope.scan(rows, columns, scope.New(elem.Addr().Interface()).Fields())

  if isSlice {
    if isPtr {
      results.Set(reflect.Append(results, elem.Addr()))
    } else {
      results.Set(reflect.Append(results, elem))
    }
  }
}

The core statement of this code is scope.scan , take a look at the definition of this method:

func (scope *Scope) scan(rows *sql.Rows, columns []string, fields []*Field) {
    var (
        ignored            interface{}
        values             = make([]interface{}, len(columns))
        selectFields       []*Field
        selectedColumnsMap = map[string]int{}
        resetFields        = map[int]*Field{}
    )

    for index, column := range columns {
        values[index] = &ignored

        selectFields = fields
        offset := 0
        if idx, ok := selectedColumnsMap[column]; ok {
            offset = idx + 1
            selectFields = selectFields[offset:]
        }

        for fieldIndex, field := range selectFields {
            if field.DBName == column {
                if field.Field.Kind() == reflect.Ptr {
                    values[index] = field.Field.Addr().Interface()
                } else {
                    reflectValue := reflect.New(reflect.PtrTo(field.Struct.Type))
                    reflectValue.Elem().Set(field.Field.Addr())
                    values[index] = reflectValue.Interface()
                    resetFields[index] = field
                }

                selectedColumnsMap[column] = offset + fieldIndex

                if field.IsNormal {
                    break
                }
            }
        }
    }

    scope.Err(rows.Scan(values...))

    for index, field := range resetFields {
        if v := reflect.ValueOf(values[index]).Elem().Elem(); v.IsValid() {
            field.Field.Set(v)
        }
    }
}

As its name implies, it is actually called rows.Scan(values...), copy the queried data to the corresponding fields

Thus, we understand the main process of query

Focus on the process and skip the details of building SQL statements to take a closer look at the prepareQuerySQL method

Build query SQL statement

func (scope *Scope) prepareQuerySQL() {
    if scope.Search.raw {
        scope.Raw(scope.CombinedConditionSql())
    } else {
        scope.Raw(fmt.Sprintf("SELECT %v FROM %v %v", scope.selectSQL(), scope.QuotedTableName(), scope.CombinedConditionSql()))
    }
    return
}

Used in internal branches scope.Raw , take a look at its implementation:

// Raw set raw sql
func (scope *Scope) Raw(sql string) *Scope {
    scope.SQL = strings.Replace(sql, "$$$", "?", -1)
    return scope
}

Its function is to assign the obtained sql statement to the scope.SQL Field, in which all $$$are replaced with

Back to prepareQuerySQL, the important part is the Raw parameter
A better understanding of the second half of if is to build a SELECT expression

SELECT expression needs three variables, field name, table name and condition

Take a look at each one

func (scope *Scope) selectSQL() string {
    if len(scope.Search.selects) == 0 {
        if len(scope.Search.joinConditions) > 0 {
            return fmt.Sprintf("%v.*", scope.QuotedTableName())
        }
        return "*"
    }
    return scope.buildSelectQuery(scope.Search.selects)
}

func (scope *Scope) buildSelectQuery(clause map[string]interface{}) (str string) {
    switch value := clause["query"].(type) {
    case string:
        str = value
    case []string:
        str = strings.Join(value, ", ")
    }

    args := clause["args"].([]interface{})
    replacements := []string{}
    for _, arg := range args {
        switch reflect.ValueOf(arg).Kind() {
        case reflect.Slice:
            values := reflect.ValueOf(arg)
            var tempMarks []string
            for i := 0; i < values.Len(); i++ {
                tempMarks = append(tempMarks, scope.AddToVars(values.Index(i).Interface()))
            }
            replacements = append(replacements, strings.Join(tempMarks, ","))
        default:
            if valuer, ok := interface{}(arg).(driver.Valuer); ok {
                arg, _ = valuer.Value()
            }
            replacements = append(replacements, scope.AddToVars(arg))
        }
    }

    buff := bytes.NewBuffer([]byte{})
    i := 0
    for pos, char := range str {
        if str[pos] == '?' {
            buff.WriteString(replacements[i])
            i++
        } else {
            buff.WriteRune(char)
        }
    }

    str = buff.String()

    return
}

When scope.Search.selects When it is empty, it is relatively simple
As long as there is a linked table query, return table. * or *

Build select query is based on scope.Search.selects Build query field name

The first half is clear at a glance

switch value := clause["query"].(type) {
case string:
  str = value
case []string:
  str = strings.Join(value, ", ")
}

The focus is on how to deal with parameters, that is, the second half of the code

args := clause["args"].([]interface{})
replacements := []string{}
for _, arg := range args {
  switch reflect.ValueOf(arg).Kind() {
  case reflect.Slice:
    values := reflect.ValueOf(arg)
    var tempMarks []string
    for i := 0; i < values.Len(); i++ {
      tempMarks = append(tempMarks, scope.AddToVars(values.Index(i).Interface()))
    }
    replacements = append(replacements, strings.Join(tempMarks, ","))
  default:
    if valuer, ok := interface{}(arg).(driver.Valuer); ok {
      arg, _ = valuer.Value()
    }
    replacements = append(replacements, scope.AddToVars(arg))
  }
}

buff := bytes.NewBuffer([]byte{})
i := 0
for pos, char := range str {
  if str[pos] == '?' {
    buff.WriteString(replacements[i])
    i++
  } else {
    buff.WriteRune(char)
  }
}

The main process is to traverse args: = clause ["args"]. ([] interface {}),
A replacement slice is created, and then all of the?,
Replace with the corresponding field

At this point, the process of building the SELECT field is over

The process of getting the table name is relatively simple. Show the code directly:

// QuotedTableName return quoted table name
func (scope *Scope) QuotedTableName() (name string) {
    if scope.search != nil && len(scope.Search.tableName) > 0 {
        if strings.Contains(scope.Search.tableName, " ") {
            return scope.Search.tableName
        }
        return scope.Quote(scope.Search.tableName)
    }

    return scope.Quote(scope.TableName())
}

Conditional statement

More attention is focused on how to build a filter condition, that is, the CombinedConditionSql method

// CombinedConditionSql return combined condition sql
func (scope *Scope) CombinedConditionSql() string {
    joinSQL := scope.joinsSQL()
    whereSQL := scope.whereSQL()
    if scope.Search.raw {
        whereSQL = strings.TrimSuffix(strings.TrimPrefix(whereSQL, "WHERE ("), ")")
    }
    return joinSQL + whereSQL + scope.groupSQL() +
        scope.havingSQL() + scope.orderSQL() + scope.limitAndOffsetSQL()
}

In short code, there are compact logic, conditional statements have many modules, and there are 6 clauses in total
Read it all, after reading it, you should be familiar with how to build conditional statements

func (scope *Scope) joinsSQL() string {
    var joinConditions []string
    for _, clause := range scope.Search.joinConditions {
        if sql := scope.buildCondition(clause, true); sql != "" {
            joinConditions = append(joinConditions, strings.TrimSuffix(strings.TrimPrefix(sql, "("), ")"))
        }
    }

    return strings.Join(joinConditions, " ") + " "
}

In the process of creating joinSQL, buildCondition is mainly used. Continue to deepen:

func (scope *Scope) buildCondition(clause map[string]interface{}, include bool) (str string) {
    var (
        quotedTableName  = scope.QuotedTableName()
        quotedPrimaryKey = scope.Quote(scope.PrimaryKey())
        equalSQL         = "="
        inSQL            = "IN"
    )

    // If building not conditions
    if !include {
        equalSQL = "<>"
        inSQL = "NOT IN"
    }

    switch value := clause["query"].(type) {
    case sql.NullInt64:
        return fmt.Sprintf("(%v.%v %s %v)", quotedTableName, quotedPrimaryKey, equalSQL, value.Int64)
    case int, int8, int16, int32, int64, uint, uint8, uint16, uint32, uint64:
        return fmt.Sprintf("(%v.%v %s %v)", quotedTableName, quotedPrimaryKey, equalSQL, value)
    case []int, []int8, []int16, []int32, []int64, []uint, []uint8, []uint16, []uint32, []uint64, []string, []interface{}:
        if !include && reflect.ValueOf(value).Len() == 0 {
            return
        }
        str = fmt.Sprintf("(%v.%v %s (?))", quotedTableName, quotedPrimaryKey, inSQL)
        clause["args"] = []interface{}{value}
    case string:
        if isNumberRegexp.MatchString(value) {
            return fmt.Sprintf("(%v.%v %s %v)", quotedTableName, quotedPrimaryKey, equalSQL, scope.AddToVars(value))
        }

        if value != "" {
            if !include {
                if comparisonRegexp.MatchString(value) {
                    str = fmt.Sprintf("NOT (%v)", value)
                } else {
                    str = fmt.Sprintf("(%v.%v NOT IN (?))", quotedTableName, scope.Quote(value))
                }
            } else {
                str = fmt.Sprintf("(%v)", value)
            }
        }
    case map[string]interface{}:
        var sqls []string
        for key, value := range value {
            if value != nil {
                sqls = append(sqls, fmt.Sprintf("(%v.%v %s %v)", quotedTableName, scope.Quote(key), equalSQL, scope.AddToVars(value)))
            } else {
                if !include {
                    sqls = append(sqls, fmt.Sprintf("(%v.%v IS NOT NULL)", quotedTableName, scope.Quote(key)))
                } else {
                    sqls = append(sqls, fmt.Sprintf("(%v.%v IS NULL)", quotedTableName, scope.Quote(key)))
                }
            }
        }
        return strings.Join(sqls, " AND ")
    case interface{}:
        var sqls []string
        newScope := scope.New(value)

        if len(newScope.Fields()) == 0 {
            scope.Err(fmt.Errorf("invalid query condition: %v", value))
            return
        }
        scopeQuotedTableName := newScope.QuotedTableName()
        for _, field := range newScope.Fields() {
            if !field.IsIgnored && !field.IsBlank {
                sqls = append(sqls, fmt.Sprintf("(%v.%v %s %v)", scopeQuotedTableName, scope.Quote(field.DBName), equalSQL, scope.AddToVars(field.Field.Interface())))
            }
        }
        return strings.Join(sqls, " AND ")
    default:
        scope.Err(fmt.Errorf("invalid query condition: %v", value))
        return
    }

    replacements := []string{}
    args := clause["args"].([]interface{})
    for _, arg := range args {
        var err error
        switch reflect.ValueOf(arg).Kind() {
        case reflect.Slice: // For where("id in (?)", []int64{1,2})
            if scanner, ok := interface{}(arg).(driver.Valuer); ok {
                arg, err = scanner.Value()
                replacements = append(replacements, scope.AddToVars(arg))
            } else if b, ok := arg.([]byte); ok {
                replacements = append(replacements, scope.AddToVars(b))
            } else if as, ok := arg.([][]interface{}); ok {
                var tempMarks []string
                for _, a := range as {
                    var arrayMarks []string
                    for _, v := range a {
                        arrayMarks = append(arrayMarks, scope.AddToVars(v))
                    }

                    if len(arrayMarks) > 0 {
                        tempMarks = append(tempMarks, fmt.Sprintf("(%v)", strings.Join(arrayMarks, ",")))
                    }
                }

                if len(tempMarks) > 0 {
                    replacements = append(replacements, strings.Join(tempMarks, ","))
                }
            } else if values := reflect.ValueOf(arg); values.Len() > 0 {
                var tempMarks []string
                for i := 0; i < values.Len(); i++ {
                    tempMarks = append(tempMarks, scope.AddToVars(values.Index(i).Interface()))
                }
                replacements = append(replacements, strings.Join(tempMarks, ","))
            } else {
                replacements = append(replacements, scope.AddToVars(Expr("NULL")))
            }
        default:
            if valuer, ok := interface{}(arg).(driver.Valuer); ok {
                arg, err = valuer.Value()
            }

            replacements = append(replacements, scope.AddToVars(arg))
        }

        if err != nil {
            scope.Err(err)
        }
    }

    buff := bytes.NewBuffer([]byte{})
    i := 0
    for _, s := range str {
        if s == '?' && len(replacements) > i {
            buff.WriteString(replacements[i])
            i++
        } else {
            buff.WriteRune(s)
        }
    }

    str = buff.String()

    return
}

At the beginning, it is a delicate choice. Based on include, it realizes not condition

var (
  quotedTableName  = scope.QuotedTableName()
  quotedPrimaryKey = scope.Quote(scope.PrimaryKey())
  equalSQL         = "="
  inSQL            = "IN"
)

// If building not conditions
if !include {
  equalSQL = "<>"
  inSQL = "NOT IN"
}

In the middle is a switch value: = clause ["query"] (type) selection
In this switch selection, most of the conditions are returned directly
The rest will build the str string variable

This will continue to the end. The code in this part is very similar to what we saw above,
It is to build replacement slices according to clause["args"],
Used to replace the "? In the str variable

Next look at the next where SQL method

func (scope *Scope) whereSQL() (sql string) {
    var (
        quotedTableName                                = scope.QuotedTableName()
        deletedAtField, hasDeletedAtField              = scope.FieldByName("DeletedAt")
        primaryConditions, andConditions, orConditions []string
    )

    if !scope.Search.Unscoped && hasDeletedAtField {
        sql := fmt.Sprintf("%v.%v IS NULL", quotedTableName, scope.Quote(deletedAtField.DBName))
        primaryConditions = append(primaryConditions, sql)
    }

    if !scope.PrimaryKeyZero() {
        for _, field := range scope.PrimaryFields() {
            sql := fmt.Sprintf("%v.%v = %v", quotedTableName, scope.Quote(field.DBName), scope.AddToVars(field.Field.Interface()))
            primaryConditions = append(primaryConditions, sql)
        }
    }

    for _, clause := range scope.Search.whereConditions {
        if sql := scope.buildCondition(clause, true); sql != "" {
            andConditions = append(andConditions, sql)
        }
    }

    for _, clause := range scope.Search.orConditions {
        if sql := scope.buildCondition(clause, true); sql != "" {
            orConditions = append(orConditions, sql)
        }
    }

    for _, clause := range scope.Search.notConditions {
        if sql := scope.buildCondition(clause, false); sql != "" {
            andConditions = append(andConditions, sql)
        }
    }

    orSQL := strings.Join(orConditions, " OR ")
    combinedSQL := strings.Join(andConditions, " AND ")
    if len(combinedSQL) > 0 {
        if len(orSQL) > 0 {
            combinedSQL = combinedSQL + " OR " + orSQL
        }
    } else {
        combinedSQL = orSQL
    }

    if len(primaryConditions) > 0 {
        sql = "WHERE " + strings.Join(primaryConditions, " AND ")
        if len(combinedSQL) > 0 {
            sql = sql + " AND (" + combinedSQL + ")"
        }
    } else if len(combinedSQL) > 0 {
        sql = "WHERE " + combinedSQL
    }
    return
}

It mainly constructs three parts: primary conditions, and conditions, or conditions

if !scope.Search.Unscoped && hasDeletedAtField {
  sql := fmt.Sprintf("%v.%v IS NULL", quotedTableName, scope.Quote(deletedAtField.DBName))
  primaryConditions = append(primaryConditions, sql)
}

if !scope.PrimaryKeyZero() {
  for _, field := range scope.PrimaryFields() {
    sql := fmt.Sprintf("%v.%v = %v", quotedTableName, scope.Quote(field.DBName), scope.AddToVars(field.Field.Interface()))
    primaryConditions = append(primaryConditions, sql)
  }
}

The first two if constructs the primaryConditions condition

for _, clause := range scope.Search.whereConditions {
  if sql := scope.buildCondition(clause, true); sql != "" {
    andConditions = append(andConditions, sql)
  }
}

for _, clause := range scope.Search.orConditions {
  if sql := scope.buildCondition(clause, true); sql != "" {
    orConditions = append(orConditions, sql)
  }
}

for _, clause := range scope.Search.notConditions {
  if sql := scope.buildCondition(clause, false); sql != "" {
    andConditions = append(andConditions, sql)
  }
}

Then three for loops use the buildCondition method
be aware scope.Search.notConditions It's in and conditions

orSQL := strings.Join(orConditions, " OR ")
combinedSQL := strings.Join(andConditions, " AND ")
if len(combinedSQL) > 0 {
  if len(orSQL) > 0 {
    combinedSQL = combinedSQL + " OR " + orSQL
  }
} else {
  combinedSQL = orSQL
}

The condition statement is generated by combining orConditions and andConditions

if len(primaryConditions) > 0 {
  sql = "WHERE " + strings.Join(primaryConditions, " AND ")
  if len(combinedSQL) > 0 {
    sql = sql + " AND (" + combinedSQL + ")"
  }
} else if len(combinedSQL) > 0 {
  sql = "WHERE " + combinedSQL
}
return

Finally, the final WHERE clause is generated with primaryConditions

Then look at the other:

func (scope *Scope) groupSQL() string {
    if len(scope.Search.group) == 0 {
        return ""
    }
    return " GROUP BY " + scope.Search.group
}

GROUP BY clause is simple and can be built directly

continue:

func (scope *Scope) havingSQL() string {
    if len(scope.Search.havingConditions) == 0 {
        return ""
    }

    var andConditions []string
    for _, clause := range scope.Search.havingConditions {
        if sql := scope.buildCondition(clause, true); sql != "" {
            andConditions = append(andConditions, sql)
        }
    }

    combinedSQL := strings.Join(andConditions, " AND ")
    if len(combinedSQL) == 0 {
        return ""
    }

    return " HAVING " + combinedSQL
}

The HAVING clause is not difficult either. After the condition is built, connect with AND, AND then add HAVING at the top

continue:

func (scope *Scope) orderSQL() string {
    if len(scope.Search.orders) == 0 || scope.Search.ignoreOrderQuery {
        return ""
    }

    var orders []string
    for _, order := range scope.Search.orders {
        if str, ok := order.(string); ok {
            orders = append(orders, scope.quoteIfPossible(str))
        } else if expr, ok := order.(*expr); ok {
            exp := expr.expr
            for _, arg := range expr.args {
                exp = strings.Replace(exp, "?", scope.AddToVars(arg), 1)
            }
            orders = append(orders, exp)
        }
    }
    return " ORDER BY " + strings.Join(orders, ",")
}

The structure is similar, traversal scope.Search.orders Slicing and order have two different types, string or expr structure
The latter deals with the case with parameters

Finally, there is a limitAndOffsetSQL method:

func (scope *Scope) limitAndOffsetSQL() string {
    return scope.Dialect().LimitAndOffsetSQL(scope.Search.limit, scope.Search.offset)
}

This directly calls the LimitAndOffsetSQL method in the specific database driver

Look at two specific implementations, one is the implementation in general, the other is the implementation in mysql

func (commonDialect) LimitAndOffsetSQL(limit, offset interface{}) (sql string) {
    if limit != nil {
        if parsedLimit, err := strconv.ParseInt(fmt.Sprint(limit), 0, 0); err == nil && parsedLimit >= 0 {
            sql += fmt.Sprintf(" LIMIT %d", parsedLimit)
        }
    }
    if offset != nil {
        if parsedOffset, err := strconv.ParseInt(fmt.Sprint(offset), 0, 0); err == nil && parsedOffset >= 0 {
            sql += fmt.Sprintf(" OFFSET %d", parsedOffset)
        }
    }
    return
}

Directly resolve limit and offset to int type, and then connect the corresponding keywords

Next, take a look at the implementation in mysql:

func (s mysql) LimitAndOffsetSQL(limit, offset interface{}) (sql string) {
    if limit != nil {
        if parsedLimit, err := strconv.ParseInt(fmt.Sprint(limit), 0, 0); err == nil && parsedLimit >= 0 {
            sql += fmt.Sprintf(" LIMIT %d", parsedLimit)

            if offset != nil {
                if parsedOffset, err := strconv.ParseInt(fmt.Sprint(offset), 0, 0); err == nil && parsedOffset >= 0 {
                    sql += fmt.Sprintf(" OFFSET %d", parsedOffset)
                }
            }
        }
    }
    return
}

The difference between the two is the nesting of offset s, which must be used together with limit in mysql

In this way, all the clauses in the combined condition SQL are finished
In fact, there is no magic, but according to different conditions, build different SQL statements

Summary

Go all the way from First to the internal details of the query. After understanding the underlying details, other similar methods will not be difficult to understand

// Take return a record that match given conditions, the order will depend on the database implementation
func (s *DB) Take(out interface{}, where ...interface{}) *DB {
    newScope := s.NewScope(out)
    newScope.Search.Limit(1)
    return newScope.inlineCondition(where...).callCallbacks(s.parent.callbacks.queries).db
}

// Last find last record that match given conditions, order by primary key
func (s *DB) Last(out interface{}, where ...interface{}) *DB {
    newScope := s.NewScope(out)
    newScope.Search.Limit(1)
    return newScope.Set("gorm:order_by_primary_key", "DESC").
        inlineCondition(where...).callCallbacks(s.parent.callbacks.queries).db
}

// Find find records that match given conditions
func (s *DB) Find(out interface{}, where ...interface{}) *DB {
    return s.NewScope(out).inlineCondition(where...).callCallbacks(s.parent.callbacks.queries).db
}

search structure

In the previous process, we only saw how the simplest query was generated
In this process, we did not carefully study how query conditions are stored

See how to use the Where method to add query criteria

// Get first matched record
db.Where("name = ?", "jinzhu").First(&user)
//// SELECT * FROM users WHERE name = 'jinzhu' limit 1;

// Get all matched records
db.Where("name = ?", "jinzhu").Find(&users)
//// SELECT * FROM users WHERE name = 'jinzhu';

The above example comes from the official document. GORM uses the chain call style to concatenate multiple Where methods or other query conditions

// Where return a new relation, filter records with given conditions, accepts `map`, `struct` or `string` as conditions, refer http://jinzhu.github.io/gorm/crud.html#query
func (s *DB) Where(query interface{}, args ...interface{}) *DB {
    return s.clone().search.Where(query, args...).db
}

Above is the code of Where method. There are many similar methods near its source code

// Or filter records that match before conditions or this one, similar to `Where`
func (s *DB) Or(query interface{}, args ...interface{}) *DB {
    return s.clone().search.Or(query, args...).db
}

// Not filter records that don't match current conditions, similar to `Where`
func (s *DB) Not(query interface{}, args ...interface{}) *DB {
    return s.clone().search.Not(query, args...).db
}

It is easy to find that the source of all this is the search object

When the structure DB is defined, a field is search:

search            *search

Definition of search

This is where query conditions are stored. Its definition is as follows:

type search struct {
    db               *DB
    whereConditions  []map[string]interface{}
    orConditions     []map[string]interface{}
    notConditions    []map[string]interface{}
    havingConditions []map[string]interface{}
    joinConditions   []map[string]interface{}
    initAttrs        []interface{}
    assignAttrs      []interface{}
    selects          map[string]interface{}
    omits            []string
    orders           []interface{}
    preload          []searchPreload
    offset           interface{}
    limit            interface{}
    group            string
    tableName        string
    raw              bool
    Unscoped         bool
    ignoreOrderQuery bool
}

type searchPreload struct {
    schema     string
    conditions []interface{}
}

There are many fields of type [] map[string]interface {} here. Combined with the previous code about condition query, you can recall that this is where all kinds of conditions are stored

Other fields such as offset and limit are easy to understand

search method

There are many methods under search. Although there are many methods, they are basically very short. In total, they are more than 100 lines

func (s *search) clone() *search {
    clone := *s
    return &clone
}

This cloning method is a little unique. It seems that I didn't do anything. Maybe I have little experience

func (s *search) Where(query interface{}, values ...interface{}) *search {
    s.whereConditions = append(s.whereConditions, map[string]interface{}{"query": query, "args": values})
    return s
}

func (s *search) Not(query interface{}, values ...interface{}) *search {
    s.notConditions = append(s.notConditions, map[string]interface{}{"query": query, "args": values})
    return s
}

func (s *search) Or(query interface{}, values ...interface{}) *search {
    s.orConditions = append(s.orConditions, map[string]interface{}{"query": query, "args": values})
    return s
}

The above methods are all built into a map with parameters and then pushed into the corresponding slice. Considering the chain call, they return to themselves

func (s *search) Attrs(attrs ...interface{}) *search {
    s.initAttrs = append(s.initAttrs, toSearchableMap(attrs...))
    return s
}

func (s *search) Assign(attrs ...interface{}) *search {
    s.assignAttrs = append(s.assignAttrs, toSearchableMap(attrs...))
    return s
}

func toSearchableMap(attrs ...interface{}) (result interface{}) {
    if len(attrs) > 1 {
        if str, ok := attrs[0].(string); ok {
            result = map[string]interface{}{str: attrs[1]}
        }
    } else if len(attrs) == 1 {
        if attr, ok := attrs[0].(map[string]interface{}); ok {
            result = attr
        }

        if attr, ok := attrs[0].(interface{}); ok {
            result = attr
        }
    }
    return
}

The two methods are similar and use the toSearchableMap transformation parameter

func (s *search) Order(value interface{}, reorder ...bool) *search {
    if len(reorder) > 0 && reorder[0] {
        s.orders = []interface{}{}
    }

    if value != nil && value != "" {
        s.orders = append(s.orders, value)
    }
    return s
}

It may be a bit confusing to see this. You can get an explanation from the documents and comments

// Order specify order when retrieve records from database, set reorder to `true` to overwrite defined conditions
//     db.Order("name DESC")
//     db.Order("name DESC", true) // reorder
//     db.Order(gorm.Expr("name = ? DESC", "first")) // sql expression
func (s *DB) Order(value interface{}, reorder ...bool) *DB {
    return s.clone().search.Order(value, reorder...).db
}

The second parameter is used to determine whether to override the previous sorting conditions

It may be a little strange why reorder is a variable parameter, I don't know for compatibility or historical legacy

Another point is that we can't understand [] interface {}}, which can be divided into two parts. In fact, [] interface {} is a type, and {} constructs an empty instance of this type

func (s *search) Select(query interface{}, args ...interface{}) *search {
    s.selects = map[string]interface{}{"query": query, "args": args}
    return s
}

func (s *search) Omit(columns ...string) *search {
    s.omits = columns
    return s
}

func (s *search) Limit(limit interface{}) *search {
    s.limit = limit
    return s
}

func (s *search) Offset(offset interface{}) *search {
    s.offset = offset
    return s
}

These are replacement type, each call will only save the latest value

func (s *search) Group(query string) *search {
    s.group = s.getInterfaceAsSQL(query)
    return s
}

func (s *search) getInterfaceAsSQL(value interface{}) (str string) {
    switch value.(type) {
    case string, int, int8, int16, int32, int64, uint, uint8, uint16, uint32, uint64:
        str = fmt.Sprintf("%v", value)
    default:
        s.db.AddError(ErrInvalidSQL)
    }

    if str == "-1" {
        return ""
    }
    return
}

A feature of getInterfaceAsSQL is that - 1 resets it

func (s *search) Having(query interface{}, values ...interface{}) *search {
    if val, ok := query.(*expr); ok {
        s.havingConditions = append(s.havingConditions, map[string]interface{}{"query": val.expr, "args": val.args})
    } else {
        s.havingConditions = append(s.havingConditions, map[string]interface{}{"query": query, "args": values})
    }
    return s
}

func (s *search) Joins(query string, values ...interface{}) *search {
    s.joinConditions = append(s.joinConditions, map[string]interface{}{"query": query, "args": values})
    return s
}

In fact, this is similar to what we have seen before, so there is not much explanation

func (s *search) Preload(schema string, values ...interface{}) *search {
    var preloads []searchPreload
    for _, preload := range s.preload {
        if preload.schema != schema {
            preloads = append(preloads, preload)
        }
    }
    preloads = append(preloads, searchPreload{schema, values})
    s.preload = preloads
    return s
}

Preload needs to prevent repetition, so it will traverse the existing schema again at the beginning

func (s *search) Raw(b bool) *search {
    s.raw = b
    return s
}

func (s *search) unscoped() *search {
    s.Unscoped = true
    return s
}

func (s *search) Table(name string) *search {
    s.tableName = name
    return s
}

There is nothing special about the last few methods

Summary

The search structure is quite simple. There are more than 100 lines to define the addition method
But it's useful. Query related conditions are stored here

summary

This part mainly looks at how SQL query occurs, and explores how various query clauses are implemented in this process. At the same time, it also studies the search structure and its role