Configuration files are arguably one of the most frequently encountered parts of our use of nginx. After we configure the appropriate configuration items, we typically use the. /sbin/nginx -t command to test the configuration file for parameter errors and reload nginx.This paper will first explain the configuration of nginx configuration file with an example, then explain the parsing principle of nginx configuration file from the source code point of view.
1. Use examples
1.1 Configuration file usage example
daemon off; error_log stderr info; events { worker_connections 1024; } http { proxy_cache_path /nginx-cache levels=1:2 keys_zone=one:10m max_size=10g inactive=60m; sendfile on; keepalive_timeout 65; server { listen 8080; server_name localhost; location / { root html; } } }
The configuration rules for nginx, regardless of the specific meaning of each configuration item, are as follows:
- There are two main types of configuration items for nginx: configuration blocks and configuration items.For example, here http {} is a configuration block, and daemon off; is a configuration item;
- Each configuration item is resolved by a module that defines how the configuration item is configured, including switch flags, time, space, and general configuration items:
- Switch token: For example, the sendfile instruction here can only be followed by one parameter, which must be on or off;
- Time: For example, here proxy_cache_path directive is followed by an inactive=60m, where 60m denotes 60 minutes, which can be in y, M, w, d, h, M, s,, meaning year, month, week, day, hour, minute, second, and end tag, respectively;
- Space: For example, the max_size of the sendfile directive = 10g; where 10g denotes the size of space, and its units can be k, m, g, meaning kb, mb, and gb of the unit of space size, respectively;
- General configuration: such as root html here; followed by html is a general configuration item;
- When each module specifies its resolvable configuration items, it also specifies the number of parameters for its configuration items. Before resolving, nginx checks whether the number of each configuration item is the specified number. If not, no subsequent checks will be made.For example, keep alive_timeout 65; the number of configuration items must be 2 (the name of the configuration item here also counts);
- Each configuration item must be semicolon; it ends with a complete configuration item;
- Each parameter of each configuration item must be separated by a space, and if there are spaces in the configuration item, the escape character is used.
1.2 nginx Configuration Item Configuration Example
* For how nginx specifies how each configuration item is handled, let's use daemon off; this configuration item is an example.In nginx, each configuration item is subordinate to a module that configures the ngx_command_t structure corresponding to the reconfiguration item, which specifies properties such as the type of the configuration item, the number of parameters, how the configuration item is parsed, and so on.Here is how the daemon configuration item is configured:
static ngx_command_t ngx_core_commands[] = { {ngx_string("daemon"), NGX_MAIN_CONF | NGX_DIRECT_CONF | NGX_CONF_FLAG, ngx_conf_set_flag_slot, 0, offsetof(ngx_core_conf_t, daemon), NULL} };
The meaning of each of the above configuration items is as follows:
- ngx_string("daemon"): Specifies the name of the current configuration item, daemon;
- NGX_MAIN_CONF|NGX_DIRECT_CONF|NGX_CONF_FLAG: Here NGX_MAIN_CONF specifies that the current configuration item is the configuration of the nginx core module, that is, its configuration location must be at the outermost level of the configuration file. NGX_DIRECT_CONF specifies how the structure object generated by parsing the current configuration item is addressed in the cycle object of nginx, where it is a direct addressing, and NGX_CONF_FLAG specifiesThe parameter configuration of the current configuration item is FLAG type, that is, its value must be on or off;
- ngx_conf_set_flag_slot: This is a method that defines how to parse this configuration item and what structure objects will be generated to hold the results of the parsing.It is important to note that when nginx parses a configuration file, when a configuration item matching the current configuration file is consistent with a configuration item in a module, the method defined by the current property in that module is invoked to delegate the resolution permissions for that configuration item.In this way, each module can customize the configuration and resolution of the configuration item according to its own functional requirements.
- 0: Its property name is conf, which indicates the storage location of the current configuration object parsed by the configuration. It is generally used by the NGX_HTTP_MODULE module. Because the module is divided into main, server and location parts, which have their own separate memory pool parts, the conf property here specifies the part of the memory pool to which they belong, and generally takes three values of N.GX_HTTP_MAIN_CONF_OFFSET, NGX_HTTP_SRV_CONF_OFFSET and NGX_HTTP_LOC_CONF_OFFSET correspond to the three memory pool parts respectively, of course, a value of 0 or 0 indicates that the core module is currently in use;
- offsetof(ngx_core_conf_t, daemon): This configuration item means an offset, which is the position of the current property in the structure that defines the configuration item.Because when nginx parses a configuration item, each module typically customizes a structure to store the parsed attribute parameters, whereas the current configuration item's parameters may be an attribute in the structure, where offset specifies the location of the attribute in the structure, where daemon is located in the ngx_core_conf_t structure.Sometimes, the configuration item we parse out is a more complex structure that can no longer be represented by an offset, which can then be set to zero.
- NULL: The attribute name here is post, which is a pointer. The main purpose is that when we parse configuration data, we may need to depend on some data, where post refers to these data, and most of the time it is not necessary to specify this parameter value.
The properties of the ngx_command_t structure are explained in an example above. For the analysis of the configuration, we need to emphasize the first three parameters. When nginx parses the configuration file, for example, it parses to daemon off; this configuration item will then be looked up in all ngx_command_t structure arrays of all modules to seeWhether there is a name that matches the type (here NGX_MAIN_CONF) with the currently resolved configuration item, and if so, the current configuration item is resolved using the method pointed to by the set attribute in the ngx_command_t structure (such as the ngx_conf_set_flag_slot() method here).
2. Source Code Parsing
nginx parses configuration items mainly through the ngx_conf_parse() method of the ngx_conf_file.c file.The source code for this method is as follows:
char *ngx_conf_parse(ngx_conf_t *cf, ngx_str_t *filename) { char *rv; ngx_fd_t fd; ngx_int_t rc; ngx_buf_t buf; ngx_conf_file_t *prev, conf_file; enum { parse_file = 0, parse_block, parse_param } type; #if (NGX_SUPPRESS_WARN) fd = NGX_INVALID_FILE; prev = NULL; #endif if (filename) { /* open configuration file */ // Open Profile fd = ngx_open_file(filename->data, NGX_FILE_RDONLY, NGX_FILE_OPEN, 0); if (fd == NGX_INVALID_FILE) { ngx_conf_log_error(NGX_LOG_EMERG, cf, ngx_errno, ngx_open_file_n " \"%s\" failed", filename->data); return NGX_CONF_ERROR; } prev = cf->conf_file; cf->conf_file = &conf_file; // Check the status of the configuration file if (ngx_fd_info(fd, &cf->conf_file->file.info) == NGX_FILE_ERROR) { ngx_log_error(NGX_LOG_EMERG, cf->log, ngx_errno, ngx_fd_info_n " \"%s\" failed", filename->data); } cf->conf_file->buffer = &buf; buf.start = ngx_alloc(NGX_CONF_BUFFER, cf->log); if (buf.start == NULL) { goto failed; } buf.pos = buf.start; buf.last = buf.start; buf.end = buf.last + NGX_CONF_BUFFER; buf.temporary = 1; cf->conf_file->file.fd = fd; cf->conf_file->file.name.len = filename->len; cf->conf_file->file.name.data = filename->data; cf->conf_file->file.offset = 0; cf->conf_file->file.log = cf->log; cf->conf_file->line = 1; type = parse_file; if (ngx_dump_config #if (NGX_DEBUG) || 1 #endif ) { if (ngx_conf_add_dump(cf, filename) != NGX_OK) { goto failed; } } else { cf->conf_file->dump = NULL; } } else if (cf->conf_file->file.fd != NGX_INVALID_FILE) { type = parse_block; } else { type = parse_param; } for (;;) { /** * This is mainly to read a data unit of the configuration file for parsing, so-called data unit refers to: * 1. A configuration item ending with a semicolon; * 2. Configuration items that begin with curly brackets; * 3. Characters ending in braces; * 4. Braces after reading the entire configuration file; * For the first and second cases, after parsing the specific configuration data, the current configuration item is found in all module s * Matches this configuration item, and if it does, it is left to the handler of the module to process the configuration item, which needs to be explained here, for the configuration block, * The configuration information contained in the configuration block is further parsed in the handler of the corresponding module. * The resolution is also done through the ngx_conf_read_token() method. * For the third and fourth cases, essentially the third will only occur if handler is used. * That is, the resolution of a configuration block for representation is complete; * In the fourth case, after all the configuration items have been parsed, the main process will determine that there are no parsable characters. * And the last character will only be returned if it is an inverse brace. */ rc = ngx_conf_read_token(cf); if (rc == NGX_ERROR) { goto done; } // Go straight to the last step when the current configuration block resolution is complete if (rc == NGX_CONF_BLOCK_DONE) { if (type != parse_block) { ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "unexpected \"}\""); goto failed; } goto done; } // When the entire configuration file is parsed, go straight to the end if (rc == NGX_CONF_FILE_DONE) { if (type == parse_block) { ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "unexpected end of file, expecting \"}\""); goto failed; } goto done; } // This means that the current configuration item is followed by a left curly brace, and the contents within the curly bracket are handled by a specific handler if (rc == NGX_CONF_BLOCK_START) { if (type == parse_param) { ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "block directives are not supported " "in -g option"); goto failed; } } // Go here to indicate that the current return value is NGX_OK or NGX_CONF_BLOCK_START, that is, the specific configuration item or configuration block. // The handler here is empty by default, if not, then all configuration items will be handled by the handler if (cf->handler) { if (rc == NGX_CONF_BLOCK_START) { ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "unexpected \"{\""); goto failed; } rv = (*cf->handler)(cf, NULL, cf->handler_conf); if (rv == NGX_CONF_OK) { continue; } if (rv == NGX_CONF_ERROR) { goto failed; } ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, rv); goto failed; } // Here you begin to find which module is specified by the currently resolved configuration item and leave it to the corresponding processing method for that module to process rc = ngx_conf_handler(cf, rc); if (rc == NGX_ERROR) { goto failed; } } failed: rc = NGX_ERROR; done: // Cleanup done by parsing, such as clearing buffers, releasing file handles, etc. if (filename) { if (cf->conf_file->buffer->start) { ngx_free(cf->conf_file->buffer->start); } if (ngx_close_file(fd) == NGX_FILE_ERROR) { ngx_log_error(NGX_LOG_ALERT, cf->log, ngx_errno, ngx_close_file_n " %s failed", filename->data); rc = NGX_ERROR; } cf->conf_file = prev; } if (rc == NGX_ERROR) { return NGX_CONF_ERROR; } return NGX_CONF_OK; }
There are several ways to be aware of the above parsing process:
- ngx_conf_read_token(): The main purpose of this method is to read a token. Note that for nginx, there are two types of configuration items: configuration blocks and configuration items, such as HTTP {} and daemon off;.Both belong to token, and it is important to note that for the configuration block http, only this string belongs to token, and the individual configuration items inside it are resolved by the ngx_command_t.set() method corresponding to the HTTP configuration item, and daemon off; the whole belongs to one configuration item.
- ngx_conf_handler(): The main purpose of this method is to match the parsed token with the configuration item, that is, to look up the ngx_command_t array defined by all modules to see which one matches it, and then turn the resolution of the configuration item to the ngx_command_t.set() method for processing;
The main purpose of the ngx_conf_read_token() method is to constantly read the individual characters of the configuration file and assemble them so that the reader is interested in reading their source code.Here we will focus on the implementation of the ngx_conf_handler() method:
static ngx_int_t ngx_conf_handler(ngx_conf_t *cf, ngx_int_t last) { char *rv; void *conf, **confp; ngx_uint_t i, found; ngx_str_t *name; ngx_command_t *cmd; name = cf->args->elts; found = 0; for (i = 0; cf->cycle->modules[i]; i++) { cmd = cf->cycle->modules[i]->commands; if (cmd == NULL) { continue; } for ( /* void */ ; cmd->name.len; cmd++) { if (name->len != cmd->name.len) { continue; } // Traverse each command of each module to find which command matches the currently resolved configuration item if (ngx_strcmp(name->data, cmd->name.data) != 0) { continue; } found = 1; if (cf->cycle->modules[i]->type != NGX_CONF_MODULE && cf->cycle->modules[i]->type != cf->module_type) { continue; } /* is the directive's location right ? */ if (!(cmd->type & cf->cmd_type)) { continue; } // Check that the current command is for a configuration block and that the resolved configuration is a configuration block if (!(cmd->type & NGX_CONF_BLOCK) && last != NGX_OK) { ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "directive \"%s\" is not terminated by \";\"", name->data); return NGX_ERROR; } // Check that the command type is a configuration block and that the resolved configuration item returns an exception if it is not a configuration block if ((cmd->type & NGX_CONF_BLOCK) && last != NGX_CONF_BLOCK_START) { ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "directive \"%s\" has no opening \"{\"", name->data); return NGX_ERROR; } /* is the directive's argument count right ? */ // Here NGX_CONF_ANY means that the configuration block can have variable parameters if (!(cmd->type & NGX_CONF_ANY)) { // NGX_CONF_FLAG indicates that the value of the current configuration item must be on or off if (cmd->type & NGX_CONF_FLAG) { // The number of checks for current configuration items must be 2 if (cf->args->nelts != 2) { goto invalid; } // NGX_CONF_1MORE indicates that the number of configuration items is greater than or equal to 2 } else if (cmd->type & NGX_CONF_1MORE) { // The number of check configuration items must be greater than or equal to 2 if (cf->args->nelts < 2) { goto invalid; } // NGX_CONF_2MORE indicates that the number of configuration items must be greater than or equal to 3 } else if (cmd->type & NGX_CONF_2MORE) { // Check the number of configuration items if (cf->args->nelts < 3) { goto invalid; } // NGX_CONF_MAX_ARGS represents the maximum number of configurations nginx can use per configuration item, defaulting to 8 } else if (cf->args->nelts > NGX_CONF_MAX_ARGS) { goto invalid; // Argument_numberhere is a fixed array, indicating that the number of current configuration items must be a specific value } else if (!(cmd->type & argument_number[cf->args->nelts - 1])) { goto invalid; } } /* set up the directive's configuration context */ conf = NULL; // Find the configuration object, the NGX_DIRECT_CONF constant is used only to specify the addressing method of the configuration store, only for the core module if (cmd->type & NGX_DIRECT_CONF) { conf = ((void **) cf->ctx)[cf->cycle->modules[i]->index]; // The X_MAIN_CONF constant has two meanings. One is to specify that the context in which the instruction is used is main (or, in fact, a core module). // The second is to specify the addressing method for the configuration store. } else if (cmd->type & NGX_MAIN_CONF) { conf = &(((void **) cf->ctx)[cf->cycle->modules[i]->index]); // Apart from the core module, all other types of modules use a third configuration addressing method, which is based on the cmd->conf value // Remove the corresponding configuration from cf->ctx.For example, the optional value for cf->conf is // NGX_HTTP_MAIN_CONF_OFFSET,NGX_HTTP_SRV_CONF_OFFSET,NGX_HTTP_LOC_CONF_OFFSET， // These three HTTP configuration levels correspond to "http{}", "server{}", "location{}". } else if (cf->ctx) { confp = *(void **) ((char *) cf->ctx + cmd->conf); if (confp) { conf = confp[cf->cycle->modules[i]->ctx_index]; } } // Here, by matching the command, its corresponding handler is called, which makes the processing logic for the corresponding configuration item rv = cmd->set(cf, cmd, conf); if (rv == NGX_CONF_OK) { return NGX_OK; } if (rv == NGX_CONF_ERROR) { return NGX_ERROR; } ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "\"%s\" directive %s", name->data, rv); return NGX_ERROR; } } if (found) { ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "\"%s\" directive is not allowed here", name->data); return NGX_ERROR; } ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "unknown directive \"%s\"", name->data); return NGX_ERROR; invalid: ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "invalid number of arguments in \"%s\" directive", name->data); return NGX_ERROR; }
The ngx_conf_handler() method here accomplishes three main parts of the work:
- The first attempt is to find the ngx_command_t structure that matches the current configuration item.
- It then checks that the current configuration item's location is consistent with the definition in the ngx_command_t structure and that the number of parameters for the current configuration item is consistent with the configuration.
- Finally, if the pre-check is complete, the ngx_command_t.set() method is called to complete the resolution of the current configuration item.
Additionally, it is important to note that the above code mainly resolves a configuration item, and for resolving a configuration block, it is essentially done by calling a method of that block, because when nginx resolves a configuration item, it is done step by step, such as resolving to the initial location of the HTTP configuration block, where it records the current resolution point.So when you call the set() method of the ngx_command_t structure corresponding to http, it will continue to parse down based on where it is currently parsed, and the subsequent parsing will actually be a configuration item parsing, so the above method will be called.
3. Summary
This paper first explains the basic configuration of the nginx configuration file, then uses an example to introduce the meaning of each attribute of the ngx_command_t structure, and finally explains how nginx parses each configuration item from the source code point of view.