TOMCAT source code analysis -- start

TOMCAT source code analysis - start

Preposition

The source version of Tomcat is version 9.0.35 downloaded from the official website.

configuration file

The configuration file started by Tomcat is server.xml The start-up process is all around it, and the module structure of Tomcat can also be seen at a glance

<?xml version="1.0" encoding="UTF-8"?>
<Server port="8005" shutdown="SHUTDOWN">
  <Listener className="org.apache.catalina.startup.VersionLoggerListener" />
  <Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="on" />
  <Listener className="org.apache.catalina.core.JreMemoryLeakPreventionListener" />
  <Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />
  <Listener className="org.apache.catalina.core.ThreadLocalLeakPreventionListener" />

  
  <GlobalNamingResources>
    <Resource name="UserDatabase" auth="Container"
              type="org.apache.catalina.UserDatabase"
              description="User database that can be updated and saved"
              factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
              pathname="conf/tomcat-users.xml" />
  </GlobalNamingResources>

  <Service name="Catalina">

    <Connector port="8080" protocol="HTTP/1.1"
               connectionTimeout="20000"
               redirectPort="8443" />
    <Engine name="Catalina" defaultHost="localhost">  
      <Realm className="org.apache.catalina.realm.LockOutRealm">
        <Realm className="org.apache.catalina.realm.UserDatabaseRealm"
               resourceName="UserDatabase"/>
      </Realm>
      <Host name="localhost"  appBase="webapps"
            unpackWARs="true" autoDeploy="true">
        <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
               prefix="localhost_access_log" suffix=".txt"
               pattern="%h %l %u %t &quot;%r&quot; %s %b" />
      </Host>
    </Engine>
  </Service>
</Server>

TOMCAT module structure

The most important module of Tomcat is Container container, which is wrapped layer by layer, like "dolls", to facilitate the management of the inner layer's life cycle. Its structure is in the configuration file server.xml The XML tags in can be seen. First of all, write down the module structure of understanding this diagram, which is very helpful for understanding the sequence of the following source code startup.

life cycle

Every container life cycle in Tomcat implements an interface life cycle. Its important methods are init, start, stop, destroy, getState, etc. As you can see from its inheritance tree (only the display part), the container implements it:

Startup class

Its startup class is Bootstrap, and its startup method is main method.

    public static void main(String args[]) {
        synchronized (daemonLock) {
            if (daemon == null) {
                // Don't set daemon until init() has completed
                Bootstrap bootstrap = new Bootstrap();
                try {
                    bootstrap.init();
                } catch (Throwable t) {
                }
                daemon = bootstrap;
            } else {
                Thread.currentThread().setContextClassLoader(daemon.catalinaLoader);
            }
        }
        try {
            String command = "start";
			// ...
            } else if (command.equals("start")) {
                daemon.setAwait(true);
            	// The most important two steps -- daemon is the Bootstrap class itself
            	// Step 1: load -- after loading, it will be bound to listen to the socket port (Tomcat 8 starts to use NIO)
                daemon.load(args);
            	// Step 2: Start -- accept() will not process the request until the start
                daemon.start();
                if (null == daemon.getServer()) {
                    System.exit(1);
                }
            } 
        } catch (Throwable t) {
            
        }
    }

Load load

During the loading process, the load method of Catalina class is called through reflection

	/**
     * Load daemon.
     */
    private void load(String[] arguments) throws Exception {

        // Call the load() method
        String methodName = "load";
        Object param[];
        Class<?> paramTypes[];
        if (arguments==null || arguments.length==0) {
            paramTypes = null;
            param = null;
        } else {
            paramTypes = new Class[1];
            paramTypes[0] = arguments.getClass();
            param = new Object[1];
            param[0] = arguments;
        }
        // The `load` method of the `Catalina` class is called by reflection
        Method method =
            catalinaDaemon.getClass().getMethod(methodName, paramTypes);
        if (log.isDebugEnabled()) {
            log.debug("Calling startup class " + method);
        }
        method.invoke(catalinaDaemon, param);
    }

The loading steps are as follows:

1. Read configuration file

Read% home%/conf/server.xml The configuration file is parsed by Digester, and the configuration of the configuration file container is recursively put into the Server according to the hierarchy, which is actually the StandardServer class.

		// Set configuration source
        ConfigFileLoader.setSource(new CatalinaBaseConfigurationSource(Bootstrap.getCatalinaBaseFile(), getConfigFile()));
        File file = configFile();

        // Create and execute our Digester
        Digester digester = createStartDigester();
		// ...
        digester.push(this);
        digester.parse(inputSource);

2. Initialize the server object

getServer().init();
  • 2.1 it is found that the final method of the super lifecycle base is used for initialization (Engine, Host, Context, Wrapper, etc. are indirectly inherited from it).
public abstract class LifecycleBase implements Lifecycle {
    @Override
    public final synchronized void init() throws LifecycleException {
        // ...
        try {
            // Set status
            setStateInternal(LifecycleState.INITIALIZING, null, false);
            // Real initialization method
            initInternal();
            setStateInternal(LifecycleState.INITIALIZED, null, false);
        } catch (Throwable t) {
            handleSubClassException(t, "lifecycleBase.initFail", toString());
        }
    }

}

Looking at the definition of initInternal method, we find that it is implemented by subclasses. Here we present the advantages of Java polymorphic template method. Looking further, we will find that all container components are initialized by init() method, and finally we implement initInternal method to manage the initialization operation of our internal container.

// LifecycleBase.java
protected abstract void initInternal() throws LifecycleException;

Continue to execute the initInternal method and enter the implementation class StandardServer of the Server

    // StandardServer.java
	@Override
    protected void initInternal() throws LifecycleException {

        super.initInternal();

        // Initialize utility executor
        reconfigureUtilityExecutor(getUtilityThreadsInternal(utilityThreads));
        
        // Initialize our defined Services
        // The Server manages its own doll Service, and finally initializes the Service by using the init() method of the superclass
        for (Service service : services) {
            // init service
            service.init();
        }
    }

3. Initialize the service

In the loop at the end of the second step, enter the init method in 2.1, and then call the initInternal method of the service implementation class StandardService

// StandardService.java
    @Override
    protected void initInternal() throws LifecycleException {
		// Initialize Engine
        if (engine != null) {
            engine.init();
        }

        // Initialize actuator
        for (Executor executor : findExecutors()) {
            if (executor instanceof JmxEnabled) {
                ((JmxEnabled) executor).setDomain(getDomain());
            }
            executor.init();
        }

        // Initialize listener
        mapperListener.init();

        // Initialize Connectors (can have multiple Connectors)
        synchronized (connectorsLock) {
            for (Connector connector : connectors) {
                connector.init();
            }
        }
    }

4. Initialize engine, actuator, listener and connector

  1. Initialize engine

    It is still important to implement the initInternal method of the parent class. The configuration of the realm actually obtains the LockOutRealm.

     // StandardEngine.java
    	@Override
        protected void initInternal() throws LifecycleException {
            getRealm();
            // In this step, the 'startstopexecutor' thread pool will be created in 'ContainerBase' for use in the 'start' start stage.
            super.initInternal();
        }
     
       // ContainerBase.java
              private void reconfigureStartStopExecutor(int threads) {
                   if (threads == 1) {
                          // Use a fake executor -- a fake thread pool
                       if (!(startStopExecutor instanceof InlineExecutorService)) {
                              startStopExecutor = new InlineExecutorService();
                       }
                      } else {
                       // Delegate utility execution to the Service
                          Server server = Container.getService(this).getServer();
                          server.setUtilityThreads(threads);
                          startStopExecutor = server.getUtilityExecutor();
                      }
                  }
      
    
    
    
  2. Initialize actuator

    Source code server.xml It is not defined in. debug directly skips it.

  3. Initialize listener

    // LifecycleMBeanBase.java
       // MapperListener.java  If initInternal method is not implemented in, the method is called to the superclass
          @Override
           protected void initInternal() throws LifecycleException {
                  if (oname == null) {
                  mserver = Registry.getRegistry(null, null).getMBeanServer();
      			// Assign name to Catalina:type=Mapper
                  oname = register(this, getObjectNameKeyProperties());
              }
          }
    
    
  4. Initialize connector

        @Override
        protected void initInternal() throws LifecycleException {
    
            // Initialize adapter
            adapter = new CoyoteAdapter(this);
            protocolHandler.setAdapter(adapter);
         if (service != null) {
                protocolHandler.setUtilityExecutor(service.getServer().getUtilityExecutor());
         }
            try {
                // Initialize the protocol processor
                protocolHandler.init();
            } catch (Exception e) {
                throw new LifecycleException(
                        sm.getString("coyoteConnector.protocolHandlerInitializationFailed"), e);
            }
        }
    
    
    1. Initialize protocol processor
    // AbstractProtocol.java (Http11NioProtocol.java)
	@Override
       public void init() throws Exception {
        // Initialization end point
           String endpointName = getName();
           endpoint.setName(endpointName.substring(1, endpointName.length()-1));
           endpoint.setDomain(domain);
           endpoint.init();
       }   

	// Initialization of the end point -- in this step, the connector will let the end point bind the port
	// AbstractEndpoint.java  
	public final void init() throws Exception {
        if (bindOnInit) {
            bindWithCleanup();
            bindState = BindState.BOUND_ON_INIT;
        }
    }

	// NioEndpoint.java
 	protected void initServerSocket() throws Exception {
        if (!getUseInheritedChannel()) {
         serverSock = ServerSocketChannel.open();
            socketProperties.setProperties(serverSock.socket());
            InetSocketAddress addr = new InetSocketAddress(getAddress(), getPortWithOffset());
            // In the end, port 8080 is bound
            serverSock.socket().bind(addr,getAcceptCount());
        } else {
            
        }
        serverSock.configureBlocking(true); //mimic APR behavior
    }

‚Äč

5. Return to Bootstrap

Then, step by step, you will find that it goes out layer by layer, and finally returns to Bootstrap. Go to the next step, start().

Start start

The start procedure calls Catalina's start method through reflection.

    // Bootstrap.java
    public void start() throws Exception {
        Method method = catalinaDaemon.getClass().getMethod("start", (Class [])null);
        method.invoke(catalinaDaemon, (Object [])null);
    }
	
	// Catalina.java
    public void start() {
		// If the Server cannot be obtained, the previous load will be called. Here, it has already been loaded and will not enter this branch
        if (getServer() == null) {
            load();
        }

        // Start life cycle of official call Server
        try {
            getServer().start();
        } catch (LifecycleException e) {
            
        }
    }

The starting steps are as follows:

1. Start the server

It can be found that it is also implemented by the final start of the superclass. It can be imagined that the start method will be used for the start of all containers.

And similar to the load() method, startInternal is implemented by subclasses

   // LifecycleBase.java
	@Override
    public final synchronized void start() throws LifecycleException {
		// ...
        try {
            setStateInternal(LifecycleState.STARTING_PREP, null, false);
            // The internal logic of startInternal is implemented by subclasses
            startInternal();
        } catch (Throwable t) {

        }
    }

Enter startInternal to see the implementation. You can see that the implementation is in the standard server

// org.apache.catalina.core.StandardServer#startInternal

    @Override
    protected void startInternal() throws LifecycleException {

        // Event triggering life cycle -- START
        fireLifecycleEvent(CONFIGURE_START_EVENT, null);
        setState(LifecycleState.STARTING);

        globalNamingResources.start();

        // start the defined Service
        synchronized (servicesLock) {
            for (Service service : services) {
                service.start();
            }
        }

    }

2. Service start

Similarly, service.start(); enter the start() method of the superclass, and finally call the startInternal method of its own StandardService for implementation (the following similarities are automatically omitted).

// org.apache.catalina.core.StandardService#startInternal

    @Override
    protected void startInternal() throws LifecycleException {

        // Step 1 start the engine
        if (engine != null) {
            synchronized (engine) {
                engine.start();
            }
        }
		// Step 2 start the actuator
        synchronized (executors) {
            for (Executor executor: executors) {
                executor.start();
            }
        }
		// Step 3 start the monitor
        mapperListener.start();

        // Step 4 start the successfully loaded connector
        // So why does it fail? The most direct thing is that the port is occupied and cannot bind()
        synchronized (connectorsLock) {
            for (Connector connector: connectors) {
                // If it has already failed, don't try and start it
                if (connector.getState() != LifecycleState.FAILED) {
                    connector.start();
                }
            }
        }
    }

3. Start engine, actuator, listener and connector

  1. Start engine

    	// org.apache.catalina.core.StandardEngine#startInternal
        @Override
        protected synchronized void startInternal() throws LifecycleException {
            // Standard container startup
            // startInternal in superclass [ContainerBase] called
            super.startInternal();
        }
    
    	// org.apache.catalina.core.ContainerBase#startInternal
        @Override
        protected synchronized void startInternal() throws LifecycleException {
    		// debug found null
            Cluster cluster = getClusterInternal();
            if (cluster instanceof Lifecycle) {
                logger.info(String.format("cluster[%s] Start execution of life cycle start()", cluster.getClusterName()));
                ((Lifecycle) cluster).start();
            }
            // Get the 'LockOutRealm' loaded by the Load method`
            Realm realm = getRealmInternal();
            if (realm instanceof Lifecycle) {
                logger.info(String.format("realm-belong to container[%s] Start execution of life cycle start()", realm.getContainer().getName()));
                // The final execution is org.apache.catalina.realm.CombinedRealm#startInternal
                ((Lifecycle) realm).start();
            }
    
            // Start our child containers, if any
            // Finally, the Host [StandardHost] defined under Engine is found
            Container children[] = findChildren();
            List<Future<Void>> results = new ArrayList<>();
            for (Container child : children) {
                // Thread pool initialized when load() is used
                logger.info("Start to use thread pool to submit multiple threads to call child Container[%s]Of call Method initialization");
                results.add(startStopExecutor.submit(new StartChild(child)));
            }
    
            MultiThrowable multiThrowable = null;
    		// Use future get to block and get the initialization result of the Host
            for (Future<Void> result : results) {
                try {
                    result.get();
                } catch (Throwable e) {
                    log.error(sm.getString("containerBase.threadedStartFailed"), e);
                    if (multiThrowable == null) {
                        multiThrowable = new MultiThrowable();
                    }
                    multiThrowable.add(e);
                }
    
            }
            if (multiThrowable != null) {
                throw new LifecycleException(sm.getString("containerBase.threadedStartFailed"),
                        multiThrowable.getThrowable());
            }
    
            // Start the Valves in our pipeline (including the basic), if any
            if (pipeline instanceof Lifecycle) {
                logger.debug(String.format("Start piping pipeline[%s]Life cycle of start()", pipeline.getContainer().getName()));
                ((Lifecycle) pipeline).start();
            }
            // Declaration cycle changed to starting
            setState(LifecycleState.STARTING);
    
            // Start our thread
            if (backgroundProcessorDelay > 0) {
                monitorFuture = Container.getService(ContainerBase.this).getServer()
                        .getUtilityExecutor().scheduleWithFixedDelay(
                                new ContainerBackgroundProcessorMonitor(), 0, 60, TimeUnit.SECONDS);
            }
        }
    
    1. Start Cluster

      Get object is null, no need to start

    2. Start Realm

      With objects, skip

    3. Start subcontainer Host

      According to the above code, it will be submitted to the thread pool for multi-threaded startup, and the call method of the thread will be called

      // org.apache.catalina.core.ContainerBase.StartChild
      	// Static inner class in superclass
          private static class StartChild implements Callable<Void> {
              @Override
              public Void call() throws LifecycleException {
                  // Call the start method of the subcontainer. In this step, the subcontainer is' StandardHost '`
                  child.start();
                  //DEBUG: child: "StandardEngine[Catalina].StandardHost[localhost]"
                  return null;
              }
          }
      

      Then it enters the startInternal method through the start life cycle of StandardHost

      	// org.apache.catalina.core.StandardHost#startInternal
        	@Override
          protected synchronized void startInternal() throws LifecycleException {
      
              // Check whether there is an error in the pipeline
              // Real startup is implemented by superclass
              super.startInternal();
          }
      	
      	// Then it is implemented by ContainerBase, a familiar superclass. It has been posted in 3.1 of the start-start chapter. It will not be shown in detail here. It is mainly about the benefits of multi-threaded to start sub container -- doll mode
      	// org.apache.catalina.core.ContainerBase#startInternal
          @Override
          protected synchronized void startInternal() throws LifecycleException {
              // Start our child containers, if any
              Container children[] = findChildren();
              // Because of the server.xml The < context > container is not defined under the < host > node in, so the children array is empty this time
              List<Future<Void>> results = new ArrayList<>();
              for (Container child : children) {
                  logger.info("Start to use thread pool to submit multiple threads to call child Container[%s]Of call Method initialization");
                  results.add(startStopExecutor.submit(new StartChild(child)));
              }
      
          }
      
    4. Start pipeline

      skip

  2. Start actuator

    After multithreading the Host, click Step Out all the way to jump back to the startInternal method in StandardService.

    You can search for the following code in the StandardService class and set a breakpoint if you can't distinguish the levels.

    		//  org.apache.catalina.core.StandardService#startInternal
            synchronized (executors) {
                for (Executor executor: executors) {
                    executor.start();
                }
            }
    

    But the executors array has a length of 0, which is skipped here.

  3. lsnrctl start

    	// org.apache.catalina.mapper.MapperListener#startInternal
       @Override
        public void startInternal() throws LifecycleException {
    		// Get engine
            Engine engine = service.getContainer();
    		// Add listener into engine
            addListeners(engine);
    
            Container[] conHosts = engine.findChildren();
            for (Container conHost : conHosts) {
                Host host = (Host) conHost;
                if (!LifecycleState.NEW.equals(host.getState())) {
                    // Registering the host will register the context and wrappers
                    registerHost(host);
                }
            }
        }
    
    	// org.apache.catalina.mapper.MapperListener#addListeners
        private void addListeners(Container container) {
            // Register the listener [callback interface] into the container, life cycle, and recursively register the subcontainer
            container.addContainerListener(this);
            container.addLifecycleListener(this);
            for (Container child : container.findChildren()) {
                // Through a total of 5 layers of recursion, it is observed that the layers are as follows
                // StandardEngine[Catalina].StandardHost[localhost].StandardContext[].StandardWrapper[default]
                addListeners(child);
            }
        }
    

    It can be observed that the listener level for recursively adding child containers is: StandardEngine[Catalina].StandardHost[localhost].StandardContext[].StandardWrapper[default].

  4. Start connector

    You can org.apache.catalina.core.StandardService#startInternal Search for (Connector connector: connectors) to set a breakpoint, quickly get out of the previous step of recursion, and then enter connector.start();.

    ps: at this time, the port is occupied on the windows platform. How to kill the process occupying port 8080?

    <!-- First, find out the process occupying port 8080 -->
    netstat -ano| findstr "8080"
    <!-- Kill the process occupying port 8080[It happens to be 7400 here] -->
    taskkill /f /pid 7400
    
    // org.apache.catalina.connector.Connector#startInternal
        @Override
        protected void startInternal() throws LifecycleException {
            try {
                // Start protocol processor
                protocolHandler.start();
            } catch (Exception e) {
    
            }
        }
    
    // org.apache.coyote.AbstractProtocol#start
        @Override
        public void start() throws Exception {
    		// Start endpoint
            endpoint.start();
     
        }
    
    // org.apache.tomcat.util.net.AbstractEndpoint#start
        public final void start() throws Exception {
            startInternal();
        }
    
    // org.apache.tomcat.util.net.NioEndpoint#startInternal
       @Override
        public void startInternal() throws Exception {
            if (!running) {
                // Read configuration
                // Create worker collection
                if (getExecutor() == null) {
                    // Initialize thread pool, task queue
                    createExecutor();
                }
    		   // Create limit lock
                initializeConnectionLatch();
    
                // Creating the poller thread of NIO
                poller = new Poller();
                Thread pollerThread = new Thread(poller, getName() + "-ClientPoller");
                pollerThread.setPriority(threadPriority);
                pollerThread.setDaemon(true);
                pollerThread.start();
    		   // Actually start receiving requests from NIO port
                startAcceptorThread();
            }
        }
    
    // org.apache.tomcat.util.net.AbstractEndpoint#startAcceptorThread
        protected void startAcceptorThread() {
            acceptor = new Acceptor<>(this);
            String threadName = getName() + "-Acceptor"; // http-nio-8080-Acceptor
            acceptor.setThreadName(threadName);
            Thread t = new Thread(acceptor, threadName);
            t.setPriority(getAcceptorThreadPriority());
            t.setDaemon(getDaemon());
            t.start(); // Thread[http-nio-8080-Acceptor,5,main]
        }
    
  5. Return to BootStrap

    Keep Step Out back to BootStrap to finish the startup.

summary

  • The load process mainly completes the configuration of read instantiation containers and other components, the creation of thread pools, and the occupation of monitoring ports.
  • The start process is mainly to complete the sequence / multithreading to start the containers of each layer and start receiving the request data of the port.

harvest

  • Component modularization, in which basic operations are defined in superclasses and implementation of specific parts are defined in subclasses. The architecture is clear and hierarchical, saving a lot of code.
  • Load and start are decoupled as a whole, so it is not necessary to care about whether the required components are loaded during start-up. It's a bit like Spring's lazy loading. It reads the definition of Bean, but it doesn't need to be initialized. It can only be started when it's used (the description is not appropriate).

Keywords: Apache Java Tomcat xml

Added by anthylon on Sat, 06 Jun 2020 09:05:19 +0300