nginx startup process details

The starting process of nginx starts with the main() method of the nginx.c file, in which nginx performs operations such as parsing command line parameters, initializing module indexes, parsing configuration files, initializing modules, starting master, worker, and cache-related processes.This paper starts with the main () method and explains how it completes the whole process.

1. Process Explanation

The main () of nginx.c is a mainstream process method in which each subprocess is encapsulated and then invoked to complete initialization.The following is the workflow for the main() method:

One easily overlooked point in the above flowchart is the ngx_preinit_modules() method, which initializes the index property of each module object and sets its index to its index subscript in all module arrays.One reason for this is that when nginx subsequently saves configuration objects for each module, it also saves them on an array in the same location as the module corresponding to that configuration.In this way, each module can customize its configuration data for reading.

There is a ngx_init_cycle() method that parses the configuration file and initializes the core processes of each module throughout the initialization process.The workflow for this method is as follows:

As you can see from the above process, the ngx_init_cycle() method divides the initialization of configuration into several stages: creating module configuration objects, parsing configuration files, initializing module configuration, creating shared memory blocks, opening listening ports, and initializing modules.By slicing the whole process, nginx also provides the corresponding methods for each module to implement their own customized logic at the corresponding entry point.

2. Source explanation

2.1 nginx.c Main Process

The startup process of nginx is mainly achieved by encapsulating the various sub-processes of call amount, as follows:

int ngx_cdecl main(int argc, char *const *argv) {
  ngx_buf_t *b;
  ngx_log_t *log;
  ngx_uint_t i;
  ngx_cycle_t *cycle, init_cycle;
  ngx_conf_dump_t *cd;
  ngx_core_conf_t *ccf;

  ngx_debug_init();

  // This method initializes a chain table to store error logs for the system
  if (ngx_strerror_init() != NGX_OK) {
    return 1;
  }

  // The main purpose of this method is to parse the specific parameters passed in by the nginx runtime command line
  if (ngx_get_options(argc, argv) != NGX_OK) {
    return 1;
  }

  // Display version information for current nginx
  if (ngx_show_version) {
    ngx_show_version_info();

    // This means that if the profile is not detected, it will be returned directly because the profile needs to be parsed for profile detection.
    // So if you don't need to parse, go back directly, otherwise you'll parse the configuration file later
    if (!ngx_test_config) {
      return 0;
    }
  }

  /* TODO */ ngx_max_sockets = -1;
  // Initialize time information for nginx cache
  ngx_time_init();

#if (NGX_PCRE)
  // Initializing the relevant functions required for regular expression resolution
  ngx_regex_init();
#endif

  // Get the id of the current main process
  ngx_pid = ngx_getpid();

  // Initialize a file handle to the error log
  log = ngx_log_init(ngx_prefix);
  if (log == NULL) {
    return 1;
  }

    /* STUB */
#if (NGX_OPENSSL)
    ngx_ssl_init(log);
#endif

  /*
   * init_cycle->log is required for signal handlers and
   * ngx_process_options()
   */
  // Create a ngx_cycle_t structure, which is a core structure used by nginx to run and store various configuration information later
  ngx_memzero(&init_cycle, sizeof(ngx_cycle_t));
  init_cycle.log = log;   // Specify the log used
  ngx_cycle = &init_cycle;    // Assign the initialized object to the current global object

  // Initialize a memory pool
  init_cycle.pool = ngx_create_pool(1024, log);
  if (init_cycle.pool == NULL) {
    return 1;
  }

  // Save the parameters passed in when executing the. /nginx command into the ngx_cycle_t structure
  if (ngx_save_argv(&init_cycle, argc, argv) != NGX_OK) {
    return 1;
  }

  // The main focus here is on option values for nginx, such as the nginx root directory, error log directory, and log level for log
  if (ngx_process_options(&init_cycle) != NGX_OK) {
    return 1;
  }

  // Here we mainly get some parameters of the system, such as the number of CPU s, the maximum number of socket s, etc.
  if (ngx_os_init(log) != NGX_OK) {
    return 1;
  }

  /*
   * ngx_crc32_table_init() requires ngx_cacheline_size set in ngx_os_init()
   */

  // Initialize ngx_crc32_table_short to the value specified by ngx_crc32_table16
  if (ngx_crc32_table_init() != NGX_OK) {
    return 1;
  }

  // The main function of this method is to get the value of the NGINX environment variable and divide it according to it; or: divide each element into a socket file descriptor id,
  // In this way, the current nginx inherits these file descriptors, and then, or the attribute information of these file descriptors, sets them to the corresponding variable of the newly created ngx_listening_t structure
  if (ngx_add_inherited_sockets(&init_cycle) != NGX_OK) {
    return 1;
  }

  // The main purpose here is to assign an index property to each module in the ngx_modules array, specifying where the current module is located throughout the module.
  // Also note that there is a ctx attribute in each module, which is a multidimensional array that holds the configuration information for all modules, and the current module's configuration information is exactly in the ctx array's
  // Corresponding location
  if (ngx_preinit_modules() != NGX_OK) {
    return 1;
  }

  // This is the core way to initialize each configuration of nginx
  cycle = ngx_init_cycle(&init_cycle);
  if (cycle == NULL) {
    if (ngx_test_config) {
      ngx_log_stderr(0, "configuration file %s test failed",
                     init_cycle.conf_file.data);
    }

    return 1;
  }

  // If in test mode, print the test results of the configuration file, and if dump configuration data is required, print
  if (ngx_test_config) {
    if (!ngx_quiet_mode) {
      ngx_log_stderr(0, "configuration file %s test is successful",
                     cycle->conf_file.data);
    }

    if (ngx_dump_config) {
      cd = cycle->config_dump.elts;

      for (i = 0; i < cycle->config_dump.nelts; i++) {

        ngx_write_stdout("# configuration file ");
        (void) ngx_write_fd(ngx_stdout, cd[i].name.data,
                            cd[i].name.len);
        ngx_write_stdout(":" NGX_LINEFEED);

        b = cd[i].buffer;

        (void) ngx_write_fd(ngx_stdout, b->pos, b->last - b->pos);
        ngx_write_stdout(NGX_LINEFEED);
      }
    }

    return 0;
  }

  // Here, ngx_signal mainly records the following parameter values when the startup parameter is -s, such as reload, reopen, etc.
  if (ngx_signal) {
    // This is mainly based on the parameters after -s to send the corresponding system commands to the current master process
    return ngx_signal_process(cycle, ngx_signal);
  }

  // This is mainly to read some data of the system and print it out, which has no practical effect
  ngx_os_status(cycle->log);

  ngx_cycle = cycle;

  ccf = (ngx_core_conf_t *) ngx_get_conf(cycle->conf_ctx, ngx_core_module);

  if (ccf->master && ngx_process == NGX_PROCESS_SINGLE) {
    ngx_process = NGX_PROCESS_MASTER;
  }

#if !(NGX_WIN32)

  // The main purpose here is to set up its processing function for each signal
  if (ngx_init_signals(cycle->log) != NGX_OK) {
    return 1;
  }

  if (!ngx_inherited && ccf->daemon) {
    // If in daemon mode, create a new subprocess to start
    if (ngx_daemon(cycle->log) != NGX_OK) {
      return 1;
    }

    ngx_daemonized = 1;
  }

  if (ngx_inherited) {
    ngx_daemonized = 1;
  }

#endif

  // Rewrite pid to pid file because daemon mode creates a new subprocess
  if (ngx_create_pidfile(&ccf->pid, cycle->log) != NGX_OK) {
    return 1;
  }

  // The main point here is to point the stderr output to the log file set by the current cycle
  if (ngx_log_redirect_stderr(cycle) != NGX_OK) {
    return 1;
  }

  if (log->file->fd != ngx_stderr) {
    if (ngx_close_file(log->file->fd) == NGX_FILE_ERROR) {
      ngx_log_error(NGX_LOG_ALERT, cycle->log, ngx_errno,
                    ngx_close_file_n
                        " built-in log failed");
    }
  }

  ngx_use_stderr = 0;

  if (ngx_process == NGX_PROCESS_SINGLE) {
    // Here is the single-process model, i.e. the processing of events, maintenance of caches, etc. are handled by the master process, mainly for debugging purposes
    ngx_single_process_cycle(cycle);

  } else {
    // Here is the master-worker process model, which handles various instructions from the client.
    // And send the corresponding instructions to the worker process for processing
    ngx_master_process_cycle(cycle);
  }

  return 0;
}

The final master and worker processes start up and how they interact with each other will be explained in subsequent articles.

2.2 ngx_init_cycle() branch process

ngx_init_cycle() is an indispensable part of the initialization process of nginx. It mainly divides the startup process of nginx to achieve the entry of customized configuration of each module.The source code for this method is as follows:

ngx_cycle_t *ngx_init_cycle(ngx_cycle_t *old_cycle) {
  void *rv;
  char **senv;
  ngx_uint_t i, n;
  ngx_log_t *log;
  ngx_time_t *tp;
  ngx_conf_t conf;
  ngx_pool_t *pool;
  ngx_cycle_t *cycle, **old;
  ngx_shm_zone_t *shm_zone, *oshm_zone;
  ngx_list_part_t *part, *opart;
  ngx_open_file_t *file;
  ngx_listening_t *ls, *nls;
  ngx_core_conf_t *ccf, *old_ccf;
  ngx_core_module_t *module;
  char hostname[NGX_MAXHOSTNAMELEN];

  // Update the time zone information for the current cache
  ngx_timezone_update();

  /* force localtime update with a new timezone */

  // Time to get cache
  tp = ngx_timeofday();
  tp->sec = 0;

  // Update the time of the cache
  ngx_time_update();


  log = old_cycle->log;

  // Create a memory pool
  pool = ngx_create_pool(NGX_CYCLE_POOL_SIZE, log);
  if (pool == NULL) {
    return NULL;
  }
  pool->log = log;

  // Create a ngx_cycle_t structure object
  cycle = ngx_pcalloc(pool, sizeof(ngx_cycle_t));
  if (cycle == NULL) {
    ngx_destroy_pool(pool);
    return NULL;
  }

  // Set properties such as memory pool, log and old cycle objects
  cycle->pool = pool;
  cycle->log = log;
  cycle->old_cycle = old_cycle;

  // Set profile information
  cycle->conf_prefix.len = old_cycle->conf_prefix.len;
  cycle->conf_prefix.data = ngx_pstrdup(pool, &old_cycle->conf_prefix);
  if (cycle->conf_prefix.data == NULL) {
    ngx_destroy_pool(pool);
    return NULL;
  }

  // Set the path information for nginx
  cycle->prefix.len = old_cycle->prefix.len;
  cycle->prefix.data = ngx_pstrdup(pool, &old_cycle->prefix);
  if (cycle->prefix.data == NULL) {
    ngx_destroy_pool(pool);
    return NULL;
  }

  // Set profile information
  cycle->conf_file.len = old_cycle->conf_file.len;
  cycle->conf_file.data = ngx_pnalloc(pool, old_cycle->conf_file.len + 1);
  if (cycle->conf_file.data == NULL) {
    ngx_destroy_pool(pool);
    return NULL;
  }
  ngx_cpystrn(cycle->conf_file.data, old_cycle->conf_file.data,
              old_cycle->conf_file.len + 1);

  cycle->conf_param.len = old_cycle->conf_param.len;
  cycle->conf_param.data = ngx_pstrdup(pool, &old_cycle->conf_param);
  if (cycle->conf_param.data == NULL) {
    ngx_destroy_pool(pool);
    return NULL;
  }


  n = old_cycle->paths.nelts ? old_cycle->paths.nelts : 10;

  // Initialize the paths array for nginx
  if (ngx_array_init(&cycle->paths, pool, n, sizeof(ngx_path_t *))
      != NGX_OK) {
    ngx_destroy_pool(pool);
    return NULL;
  }

  // Initialize cycle->paths.elts array length
  ngx_memzero(cycle->paths.elts, n * sizeof(ngx_path_t *));

  // Initialize ngx_conf_dump_t structure for dump configuration file
  if (ngx_array_init(&cycle->config_dump, pool, 1, sizeof(ngx_conf_dump_t))
      != NGX_OK) {
    ngx_destroy_pool(pool);
    return NULL;
  }

  // Initialize a red-black tree for dump configuration file by ngx_str_rbtree_insert_value
  ngx_rbtree_init(&cycle->config_dump_rbtree, &cycle->config_dump_sentinel,
                  ngx_str_rbtree_insert_value);

  // Get the total number of files currently open
  if (old_cycle->open_files.part.nelts) {
    n = old_cycle->open_files.part.nelts;
    for (part = old_cycle->open_files.part.next; part; part = part->next) {
      n += part->nelts;
    }

  } else {
    n = 20;
  }

  // Initialize the list of files opened by the record, default length is 20
  if (ngx_list_init(&cycle->open_files, pool, n, sizeof(ngx_open_file_t))
      != NGX_OK) {
    ngx_destroy_pool(pool);
    return NULL;
  }


  // Calculate the number of shared memory
  if (old_cycle->shared_memory.part.nelts) {
    n = old_cycle->shared_memory.part.nelts;
    for (part = old_cycle->shared_memory.part.next; part; part = part->next) {
      n += part->nelts;
    }

  } else {
    n = 1;
  }

  // Initialize the chain table that stores shared memory information
  if (ngx_list_init(&cycle->shared_memory, pool, n, sizeof(ngx_shm_zone_t))
      != NGX_OK) {
    ngx_destroy_pool(pool);
    return NULL;
  }

  // Number of socket s to listen on
  n = old_cycle->listening.nelts ? old_cycle->listening.nelts : 10;

  // Initialize the array of socket s used to store listening
  if (ngx_array_init(&cycle->listening, pool, n, sizeof(ngx_listening_t))
      != NGX_OK) {
    ngx_destroy_pool(pool);
    return NULL;
  }

  // Initialize array elements for socket s to store listening
  ngx_memzero(cycle->listening.elts, n * sizeof(ngx_listening_t));

  // Initialize the queue to store consumable connection s
  ngx_queue_init(&cycle->reusable_connections_queue);

  // Initialize objects used to store configuration information for each module
  cycle->conf_ctx = ngx_pcalloc(pool, ngx_max_module * sizeof(void *));
  if (cycle->conf_ctx == NULL) {
    ngx_destroy_pool(pool);
    return NULL;
  }

  // Get the current hostname
  if (gethostname(hostname, NGX_MAXHOSTNAMELEN) == -1) {
    ngx_log_error(NGX_LOG_EMERG, log, ngx_errno, "gethostname() failed");
    ngx_destroy_pool(pool);
    return NULL;
  }

  /* on Linux gethostname() silently truncates name that does not fit */

  hostname[NGX_MAXHOSTNAMELEN - 1] = '\0';
  cycle->hostname.len = ngx_strlen(hostname);

  // Request memory space to store hostname
  cycle->hostname.data = ngx_pnalloc(pool, cycle->hostname.len);
  if (cycle->hostname.data == NULL) {
    ngx_destroy_pool(pool);
    return NULL;
  }

  // Convert the obtained hostname to lowercase and store it in cycle->hostname.data
  ngx_strlow(cycle->hostname.data, (u_char *) hostname, cycle->hostname.len);

  // Copy the ngx_modules array to the cycle->modules array
  if (ngx_cycle_modules(cycle) != NGX_OK) {
    ngx_destroy_pool(pool);
    return NULL;
  }


  for (i = 0; cycle->modules[i]; i++) {
    // Find the core module of nginx
    if (cycle->modules[i]->type != NGX_CORE_MODULE) {
      continue;
    }

    module = cycle->modules[i]->ctx;
    // The main purpose here is to initialize the structure of the configuration objects for each module.First determine if the create_conf() method exists for each module and, if so, call it
    // Create the corresponding conf structure and assign it to the corresponding index location of cycle->conf_ctx.
    // It is important to note that the property values of the configuration object structure created here are default values and are not initialized
    if (module->create_conf) {
      rv = module->create_conf(cycle);
      if (rv == NULL) {
        ngx_destroy_pool(pool);
        return NULL;
      }
      cycle->conf_ctx[cycle->modules[i]->index] = rv;
    }
  }


  senv = environ;


  // Create a structure to store configuration information and initialize individual parameter information
  ngx_memzero(&conf, sizeof(ngx_conf_t));
  /* STUB: init array ? */
  conf.args = ngx_array_create(pool, 10, sizeof(ngx_str_t));
  if (conf.args == NULL) {
    ngx_destroy_pool(pool);
    return NULL;
  }

  conf.temp_pool = ngx_create_pool(NGX_CYCLE_POOL_SIZE, log);
  if (conf.temp_pool == NULL) {
    ngx_destroy_pool(pool);
    return NULL;
  }


  conf.ctx = cycle->conf_ctx;
  conf.cycle = cycle;
  conf.pool = pool;
  conf.log = log;
  conf.module_type = NGX_CORE_MODULE;
  conf.cmd_type = NGX_MAIN_CONF;

#if 0
  log->log_level = NGX_LOG_DEBUG_ALL;
#endif

  if (ngx_conf_param(&conf) != NGX_CONF_OK) {
    environ = senv;
    ngx_destroy_cycle_pools(&conf);
    return NULL;
  }

  // The main purpose here is to parse the configuration file
  if (ngx_conf_parse(&conf, &cycle->conf_file) != NGX_CONF_OK) {
    environ = senv;
    ngx_destroy_cycle_pools(&conf);
    return NULL;
  }

  // If the currently executed. /nginx command is followed by the -t parameter and is not in static mode, print the parsing result of the configuration file
  if (ngx_test_config && !ngx_quiet_mode) {
    ngx_log_stderr(0, "the configuration file %s syntax is ok",
                   cycle->conf_file.data);
  }

  for (i = 0; cycle->modules[i]; i++) {
    if (cycle->modules[i]->type != NGX_CORE_MODULE) {
      continue;
    }

    module = cycle->modules[i]->ctx;
    // This is mainly to initialize the configuration information of the core module
    // For core modules, there are currently four main modules: core, errlog, regex, and events. The init_conf() method here is mainly the data in the configuration objects of these modules.
    // Set the default value because the customized value has been set during previous profile parsing
    if (module->init_conf) {
      if (module->init_conf(cycle,
                            cycle->conf_ctx[cycle->modules[i]->index])
          == NGX_CONF_ERROR) {
        environ = senv;
        ngx_destroy_cycle_pools(&conf);
        return NULL;
      }
    }
  }

  if (ngx_process == NGX_PROCESS_SIGNALLER) {
    return cycle;
  }

  // Get the configuration object for ngx_core_module
  ccf = (ngx_core_conf_t *) ngx_get_conf(cycle->conf_ctx, ngx_core_module);

  // If in test mode, create corresponding pid file
  if (ngx_test_config) {

    // Create a file that stores pid
    if (ngx_create_pidfile(&ccf->pid, log) != NGX_OK) {
      goto failed;
    }

    // Check that if the cycle object is not initialized, a pid file is created and the old pid file is deleted
  } else if (!ngx_is_init_cycle(old_cycle)) {

    /*
     * we do not create the pid file in the first ngx_init_cycle() call
     * because we need to write the demonized process pid
     */

    old_ccf = (ngx_core_conf_t *) ngx_get_conf(old_cycle->conf_ctx,
                                               ngx_core_module);
    if (ccf->pid.len != old_ccf->pid.len
        || ngx_strcmp(ccf->pid.data, old_ccf->pid.data) != 0) {
      /* new pid file name */

      if (ngx_create_pidfile(&ccf->pid, log) != NGX_OK) {
        goto failed;
      }

      ngx_delete_pidfile(old_cycle);
    }
  }

  // The main thing here is to check if the lock file has permission to operate
  if (ngx_test_lockfile(cycle->lock_file.data, log) != NGX_OK) {
    goto failed;
  }

  // The main thing here is to check that each file path of the configuration exists, create the folder if it does not exist, and check that the current user has the appropriate permissions to operate on it.
  // If not, elevate the current user's operational privileges
  if (ngx_create_paths(cycle, ccf->user) != NGX_OK) {
    goto failed;
  }

  // The main point here is to open a default error log configuration file if no error log file address is specified
  if (ngx_log_open_default(cycle) != NGX_OK) {
    goto failed;
  }

  /* open the new files */

  part = &cycle->open_files.part;
  file = part->elts;

  for (i = 0; /* void */ ; i++) {

    if (i >= part->nelts) {
      if (part->next == NULL) {
        break;
      }
      part = part->next;
      file = part->elts;
      i = 0;
    }

    if (file[i].name.len == 0) {
      continue;
    }

    // Open the specified file, such as access log, etc.
    file[i].fd = ngx_open_file(file[i].name.data,
                               NGX_FILE_APPEND,
                               NGX_FILE_CREATE_OR_OPEN,
                               NGX_FILE_DEFAULT_ACCESS);

    ngx_log_debug3(NGX_LOG_DEBUG_CORE, log, 0,
                   "log: %p %d \"%s\"",
                   &file[i], file[i].fd, file[i].name.data);

    if (file[i].fd == NGX_INVALID_FILE) {
      ngx_log_error(NGX_LOG_EMERG, log, ngx_errno,
                    ngx_open_file_n
                        " \"%s\" failed",
                    file[i].name.data);
      goto failed;
    }

#if !(NGX_WIN32)
    // The main purpose here is to set up a closed listening event for the created file, that is, to close the file when the current process exits
    if (fcntl(file[i].fd, F_SETFD, FD_CLOEXEC) == -1) {
      ngx_log_error(NGX_LOG_EMERG, log, ngx_errno,
                    "fcntl(FD_CLOEXEC) \"%s\" failed",
                    file[i].name.data);
      goto failed;
    }
#endif
  }

  cycle->log = &cycle->new_log;
  pool->log = &cycle->new_log;


  /* create shared memory */

  part = &cycle->shared_memory.part;
  shm_zone = part->elts;

  for (i = 0; /* void */ ; i++) {

    if (i >= part->nelts) {
      if (part->next == NULL) {
        break;
      }
      part = part->next;
      shm_zone = part->elts;
      i = 0;
    }

    if (shm_zone[i].shm.size == 0) {
      ngx_log_error(NGX_LOG_EMERG, log, 0,
                    "zero size shared memory zone \"%V\"",
                    &shm_zone[i].shm.name);
      goto failed;
    }

    shm_zone[i].shm.log = cycle->log;

    opart = &old_cycle->shared_memory.part;
    oshm_zone = opart->elts;

    for (n = 0; /* void */ ; n++) {

      if (n >= opart->nelts) {
        if (opart->next == NULL) {
          break;
        }
        opart = opart->next;
        oshm_zone = opart->elts;
        n = 0;
      }

      if (shm_zone[i].shm.name.len != oshm_zone[n].shm.name.len) {
        continue;
      }

      if (ngx_strncmp(shm_zone[i].shm.name.data,
                      oshm_zone[n].shm.name.data,
                      shm_zone[i].shm.name.len)
          != 0) {
        continue;
      }

      // The three if judgments here are mainly to find out that the current shared memory is exactly the same as the shared memory name and size in the old cycle and directly reuse the old shared memory
      if (shm_zone[i].tag == oshm_zone[n].tag
          && shm_zone[i].shm.size == oshm_zone[n].shm.size
          && !shm_zone[i].noreuse) {
        shm_zone[i].shm.addr = oshm_zone[n].shm.addr;
#if (NGX_WIN32)
        shm_zone[i].shm.handle = oshm_zone[n].shm.handle;
#endif

        // Reinitialize old shared memory after reuse
        if (shm_zone[i].init(&shm_zone[i], oshm_zone[n].data)
            != NGX_OK) {
          goto failed;
        }

        goto shm_zone_found;
      }

      // If not found, release old shared memory
      ngx_shm_free(&oshm_zone[n].shm);

      break;
    }

    // If you go here and there is no old shared memory to reuse, you will reapply the memory for the current shared memory
    if (ngx_shm_alloc(&shm_zone[i].shm) != NGX_OK) {
      goto failed;
    }

    // The main purpose here is to initialize pool information for shared memory
    if (ngx_init_zone_pool(cycle, &shm_zone[i]) != NGX_OK) {
      goto failed;
    }

    // Initialize shared memory for new requests
    if (shm_zone[i].init(&shm_zone[i], NULL) != NGX_OK) {
      goto failed;
    }

    shm_zone_found:

    continue;
  }


  /* handle the listening sockets */

  if (old_cycle->listening.nelts) {
    ls = old_cycle->listening.elts;
    for (i = 0; i < old_cycle->listening.nelts; i++) {
      ls[i].remain = 0;
    }

    nls = cycle->listening.elts;
    for (n = 0; n < cycle->listening.nelts; n++) {

      for (i = 0; i < old_cycle->listening.nelts; i++) {
        if (ls[i].ignore) {
          continue;
        }

        // The front is set to 0, so you won't walk into the if block
        if (ls[i].remain) {
          continue;
        }

        if (ls[i].type != nls[n].type) {
          continue;
        }

        if (ngx_cmp_sockaddr(nls[n].sockaddr, nls[n].socklen,
                             ls[i].sockaddr, ls[i].socklen, 1)
            == NGX_OK) {
          // This is mainly about reusing old file handles for listening
          nls[n].fd = ls[i].fd;
          nls[n].previous = &ls[i];
          ls[i].remain = 1;

          // If the backlog queue size of the new ngx_listening_s is not equal to the size of the old backlog queue, this will set its listen field to 1,
          // The listen field primarily means that the current handle is being listened on
          if (ls[i].backlog != nls[n].backlog) {
            nls[n].listen = 1;
          }

// This is mainly to set deferred_accept and accept_filter attributes of the new listening ngx_listening_s structure when transmitting TCP data.
// If deferred_accept is set, client connections will not be moved from the backlog queue until the client data is actually received before a TCP three-time handshake occurs
// To the accept queue, which is handled by NGINX, you can significantly reduce the resource consumption of nginx
#if (NGX_HAVE_DEFERRED_ACCEPT && defined SO_ACCEPTFILTER)

          /*
           * FreeBSD, except the most recent versions,
           * could not remove accept filter
           */
          nls[n].deferred_accept = ls[i].deferred_accept;

          if (ls[i].accept_filter && nls[n].accept_filter) {
              if (ngx_strcmp(ls[i].accept_filter,
                             nls[n].accept_filter)
                  != 0)
              {
                  nls[n].delete_deferred = 1;
                  nls[n].add_deferred = 1;
              }

          } else if (ls[i].accept_filter) {
              nls[n].delete_deferred = 1;

          } else if (nls[n].accept_filter) {
              nls[n].add_deferred = 1;
          }
#endif

#if (NGX_HAVE_DEFERRED_ACCEPT && defined TCP_DEFER_ACCEPT)

          if (ls[i].deferred_accept && !nls[n].deferred_accept) {
              nls[n].delete_deferred = 1;

          } else if (ls[i].deferred_accept != nls[n].deferred_accept)
          {
              nls[n].add_deferred = 1;
          }
#endif

// Set reuseport property value
#if (NGX_HAVE_REUSEPORT)
          if (nls[n].reuseport && !ls[i].reuseport) {
            nls[n].add_reuseport = 1;
          }
#endif

          break;
        }
      }

      if (nls[n].fd == (ngx_socket_t) -1) {
        nls[n].open = 1;
#if (NGX_HAVE_DEFERRED_ACCEPT && defined SO_ACCEPTFILTER)
        if (nls[n].accept_filter) {
            nls[n].add_deferred = 1;
        }
#endif
#if (NGX_HAVE_DEFERRED_ACCEPT && defined TCP_DEFER_ACCEPT)
        if (nls[n].deferred_accept) {
            nls[n].add_deferred = 1;
        }
#endif
      }
    }

  } else {
    ls = cycle->listening.elts;
    for (i = 0; i < cycle->listening.nelts; i++) {
      ls[i].open = 1;
#if (NGX_HAVE_DEFERRED_ACCEPT && defined SO_ACCEPTFILTER)
      if (ls[i].accept_filter) {
          ls[i].add_deferred = 1;
      }
#endif
#if (NGX_HAVE_DEFERRED_ACCEPT && defined TCP_DEFER_ACCEPT)
      if (ls[i].deferred_accept) {
          ls[i].add_deferred = 1;
      }
#endif
    }
  }

  // The main thing here is to open each listening port
  if (ngx_open_listening_sockets(cycle) != NGX_OK) {
    goto failed;
  }

  if (!ngx_test_config) {
    // The main purpose here is to set some TCP-related properties for each socket
    ngx_configure_listening_sockets(cycle);
  }


  /* commit the new cycle configuration */

  if (!ngx_use_stderr) {
    (void) ngx_log_redirect_stderr(cycle);
  }

  pool->log = cycle->log;

  // This calls the init_module() method of each module to initialize the current module
  if (ngx_init_modules(cycle) != NGX_OK) {
    /* fatal */
    exit(1);
  }


  /* close and delete stuff that lefts from an old cycle */

  /* free the unnecessary shared memory */

  opart = &old_cycle->shared_memory.part;
  oshm_zone = opart->elts;

  for (i = 0; /* void */ ; i++) {

    if (i >= opart->nelts) {
      if (opart->next == NULL) {
        goto old_shm_zone_done;
      }
      opart = opart->next;
      oshm_zone = opart->elts;
      i = 0;
    }

    part = &cycle->shared_memory.part;
    shm_zone = part->elts;

    for (n = 0; /* void */ ; n++) {

      if (n >= part->nelts) {
        if (part->next == NULL) {
          break;
        }
        part = part->next;
        shm_zone = part->elts;
        n = 0;
      }

      if (oshm_zone[i].shm.name.len == shm_zone[n].shm.name.len
          && ngx_strncmp(oshm_zone[i].shm.name.data,
                         shm_zone[n].shm.name.data,
                         oshm_zone[i].shm.name.len)
             == 0) {
        goto live_shm_zone;
      }
    }

    // The main thing here is to free up old shared memory
    ngx_shm_free(&oshm_zone[i].shm);

    live_shm_zone:

    continue;
  }

  old_shm_zone_done:


  /* close the unnecessary listening sockets */

  ls = old_cycle->listening.elts;
  for (i = 0; i < old_cycle->listening.nelts; i++) {

    if (ls[i].remain || ls[i].fd == (ngx_socket_t) -1) {
      continue;
    }

    // Turn off old listened sockets
    if (ngx_close_socket(ls[i].fd) == -1) {
      ngx_log_error(NGX_LOG_EMERG, log, ngx_socket_errno,
                    ngx_close_socket_n
                        " listening socket on %V failed",
                    &ls[i].addr_text);
    }

#if (NGX_HAVE_UNIX_DOMAIN)

    if (ls[i].sockaddr->sa_family == AF_UNIX) {
      u_char *name;

      name = ls[i].addr_text.data + sizeof("unix:") - 1;

      ngx_log_error(NGX_LOG_WARN, cycle->log, 0,
                    "deleting socket %s", name);

      if (ngx_delete_file(name) == NGX_FILE_ERROR) {
        ngx_log_error(NGX_LOG_EMERG, cycle->log, ngx_socket_errno,
                      ngx_delete_file_n
                          " %s failed", name);
      }
    }

#endif
  }


  /* close the unnecessary open files */

  part = &old_cycle->open_files.part;
  file = part->elts;

  for (i = 0; /* void */ ; i++) {

    if (i >= part->nelts) {
      if (part->next == NULL) {
        break;
      }
      part = part->next;
      file = part->elts;
      i = 0;
    }

    if (file[i].fd == NGX_INVALID_FILE || file[i].fd == ngx_stderr) {
      continue;
    }

    // Turn off old invalid file path descriptors
    if (ngx_close_file(file[i].fd) == NGX_FILE_ERROR) {
      ngx_log_error(NGX_LOG_EMERG, log, ngx_errno,
                    ngx_close_file_n
                        " \"%s\" failed",
                    file[i].name.data);
    }
  }

  // Destroy temporary memory pool
  ngx_destroy_pool(conf.temp_pool);

  if (ngx_process == NGX_PROCESS_MASTER || ngx_is_init_cycle(old_cycle)) {

    ngx_destroy_pool(old_cycle->pool);
    cycle->old_cycle = NULL;

    return cycle;
  }


  if (ngx_temp_pool == NULL) {
    ngx_temp_pool = ngx_create_pool(128, cycle->log);
    if (ngx_temp_pool == NULL) {
      ngx_log_error(NGX_LOG_EMERG, cycle->log, 0,
                    "could not create ngx_temp_pool");
      exit(1);
    }

    n = 10;

    if (ngx_array_init(&ngx_old_cycles, ngx_temp_pool, n,
                       sizeof(ngx_cycle_t *))
        != NGX_OK) {
      exit(1);
    }

    ngx_memzero(ngx_old_cycles.elts, n * sizeof(ngx_cycle_t *));

    // Clean up the old cycle object by putting it in the event queue and cleaning it up regularly
    ngx_cleaner_event.handler = ngx_clean_old_cycles;
    ngx_cleaner_event.log = cycle->log;
    ngx_cleaner_event.data = &dumb;
    dumb.fd = (ngx_socket_t) -1;
  }

  ngx_temp_pool->log = cycle->log;

  old = ngx_array_push(&ngx_old_cycles);
  if (old == NULL) {
    exit(1);
  }
  *old = old_cycle;

  if (!ngx_cleaner_event.timer_set) {
    ngx_add_timer(&ngx_cleaner_event, 30000);
    ngx_cleaner_event.timer_set = 1;
  }

  return cycle;


  failed:

  if (!ngx_is_init_cycle(old_cycle)) {
    old_ccf = (ngx_core_conf_t *) ngx_get_conf(old_cycle->conf_ctx,
                                               ngx_core_module);
    if (old_ccf->environment) {
      environ = old_ccf->environment;
    }
  }

  /* rollback the new cycle configuration */

  part = &cycle->open_files.part;
  file = part->elts;

  for (i = 0; /* void */ ; i++) {

    if (i >= part->nelts) {
      if (part->next == NULL) {
        break;
      }
      part = part->next;
      file = part->elts;
      i = 0;
    }

    if (file[i].fd == NGX_INVALID_FILE || file[i].fd == ngx_stderr) {
      continue;
    }

    // Close File Path Descriptor
    if (ngx_close_file(file[i].fd) == NGX_FILE_ERROR) {
      ngx_log_error(NGX_LOG_EMERG, log, ngx_errno,
                    ngx_close_file_n
                        " \"%s\" failed",
                    file[i].name.data);
    }
  }

  if (ngx_test_config) {
    ngx_destroy_cycle_pools(&conf);
    return NULL;
  }

  ls = cycle->listening.elts;
  for (i = 0; i < cycle->listening.nelts; i++) {
    if (ls[i].fd == (ngx_socket_t) -1 || !ls[i].open) {
      continue;
    }

    // Close listening socket handle
    if (ngx_close_socket(ls[i].fd) == -1) {
      ngx_log_error(NGX_LOG_EMERG, log, ngx_socket_errno,
                    ngx_close_socket_n
                        " %V failed",
                    &ls[i].addr_text);
    }
  }

  ngx_destroy_cycle_pools(&conf);

  return NULL;
}

3. Summary

This paper first explains the main process of nginx and the working principle of ngx_init_cycle() configuration process, then makes an in-depth analysis of these two processes from the source level.

Keywords: Programming Nginx socket Attribute Linux

Added by 01706 on Thu, 05 Dec 2019 13:57:26 +0200