1. Preface

When we encounter online problems and need to add a log to help locate them, we usually need to change the code to publish online in order to output the log, which is more cumbersome.

Think: Can you change the code on the server to make it work in real time? The answer is yes!

Starting with java5, a new Java has been added to jdk. Lang.instrument. Instrumentation class, which provides an api to reload a class file of a class at run time.
Here are some of its main api:

public interface Instrumentation {
/**
     * By adding a transformer Transformer, all subsequent class loads will be blocked by Transformer.
     * ClassFileTransformer Class is an interface that needs to be implemented when used. It has only one method, which passes information about the class and returns a byte code file of the converted class.
     */
 	void addTransformer(ClassFileTransformer transformer, boolean canRetransform);    

 /**
     * Re-trigger class loading for classes already loaded by the JVM. Use the Transformer registered above.
     * This method can modify the method body, constant pool, and attribute values, but it cannot add, delete, rename attributes or methods, or modify the signature of methods.
     */
    void retransformClasses(Class<?>... classes) throws UnmodifiableClassException;
    
/**
   *This method is used to replace the definition of a class without referencing existing class file bytes, as was done when recompiling from source code for repair and continued debugging.
   *retransformClasses should be used where existing class file bytes are to be converted, such as in byte code instrumentation.
   *This method can modify the method body, constant pool, and attribute values, but it cannot add, delete, rename attributes or methods, or modify the signature of methods.
   */
	void redefineClasses(ClassDefinition... definitions)throws  ClassNotFoundException, UnmodifiableClassException;

    /**
     * Gets the size of an object
     */
    long getObjectSize(Object objectToSize);
    
    /**
     * Add a jar to the classpath of the bootstrap classloader
     */
    void appendToBootstrapClassLoaderSearch(JarFile jarfile);
    
    /**
     * Gets all class objects currently loaded by the JVM
     */
    Class[] getAllLoadedClasses();
}

AddiTransformer allows you to join a converter that intercepts class-loaded events and returns the converted new byte codes, which can trigger class reload events via redefineClasses or retransformClasses. By combining these methods, you can achieve the goal mentioned at the beginning of the article of not modifying the code to make it work in real time.

2. Java Agent

By manipulating Instrumentation's api, you can make simple modifications to individual classes without restarting the service. Instrumentation is an interface whose implementation class InstrumentationImpl has only one private construction method.

How do I get this object? There are two ways to get an Instrumentation object:

• The Instrumentation object is passed through the agent's premain method when the jvm is started.
• After the JVM is started, the agent is loaded through the mechanism provided by the jvm, and the Instrumentation object is passed through the agent's agent main method.

3. Loading agent at java startup to get Instrumentation object practice

Write and compile an agent class. Class file, then type it into a jar package, and specify the jar package location in the jvm startup parameters, as follows:
1. Create an agent class and a premain method whose parameters are fixed.

public class preMainAgentClz {
    private static Instrumentation instrumentation;
    public static void premain(String agentArgs, Instrumentation inst) {
        instrumentation = inst;
        System.err.println("com.agent.demo1.preMainAgentClz I am here main Start before startup");
    }
}

2. Specify the location of the premain method (either way, just set one)
Mode 1) Create and edit resources/META-INF/MANIFEST.MF file, which is packaged together when jar packages are made

Premain-Class: com.hexuan.agent.demo1.preMainAgentClz #Location of the class where the premain method is located

Mode 2) If it is a maven project, in pom.xml join

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-jar-plugin</artifactId>
    <configuration>
        <archive>
            <manifest>
                <addClasspath>true</addClasspath>
            </manifest>
            <manifestEntries>
                <Premain-Class>com.hexuan.agent.demo1.preMainAgentClz</Premain-Class>
                <Agent-Class>com.hexuan.agent.demo1.agentMainAgentClz</Agent-Class>
            </manifestEntries>
        </archive>
    </configuration>
</plugin>

3. If it is configured in a pom, a direct maven package is good. If it is MANIFEST. The MF file specifies that the class containing premain is compiled into a class file and MANIFEEST. MF packages jar with a file.
4. Specify the agent location at startup, add the -javaagent parameter to the jvm startup parameters, and specify the jar file location.

-javaagent:/Users/IdeaProjects/acfun_WorkSpace/java-agent-demo/target/java-agent-demo-1.0-SNAPSHOT.jar

5. Start java, the premain method of the agent will be executed before the main method.

4. Loading agent as attach after java startup

The way and steps to load an agent at the start of a java process are described above, by which the specified class is replaced before starting. However, to implement the debugging online code mentioned at the beginning of this article, we need to restart the jvm after modifying the class file and set the -javaagent parameter, which is obviously not what we want most. As mentioned above, we can load agents through the mechanism provided by jvm after jvm is started, which means we can load agents at any time and then replace class files. This mechanism is jdk's attach api.

The Attach API is a set of extended APIs provided by Sun Corporation for the Attach proxy tool program to the target JVM. With it, developers can easily monitor a JVM and run an additional proxy program. The Sun JVM Attach API function is very simple, providing only the following functions:

• List all current JVM instance descriptions
• Attach goes to one of the JVM s and establishes the communication pipeline
• Let target JVM load Agent

The corresponding code location for Attach Api is at com. Sun. Tools. The attach package, which contains a VirtualMachine-like class, has two more important methods:

/**
  *Pass a process number as a parameter and return the vm object of the target jvm process.
  *This method is actually a bridge for instruction transfer between JVM processes, and the bottom layer communicates through socket s.
  *JVM A You can send some instructions to JVM B, and after B receives the instructions, it can execute the corresponding logic
  * For example, jstack, jcmd, jps, etc., which are often used on the command line, are implemented based on this mechanism
  **/
public static VirtualMachine attach(String var0) throws AttachNotSupportedException, IOException 

/**
  *This method allows us to pass the jar file address corresponding to the agent as a parameter to the target jvm
  *The target jvm will load this agent when it receives this command
  **/
public void loadAgent(String var1) throws AgentLoadException, AgentInitializationException, IOException

Obviously, we can create a java process, attach it to the corresponding jvm, and load the agent, and our classes will be successfully replaced when the agent loads.

5. How to get new class files

Instrumentation operates on. The class file, which we don't understand for our developers. Class file, not to mention modifying it directly. Consider the online code debugging scenario mentioned at the beginning of the article. We know how to replace classes, but how to get new ones. What about class files?

Mode 1: Modify offline. java file - > compile. class file -->upload to online machine -->instrument

Mode 2: Online. class Old File-->Decompiled. java files-->modify java files-->compile. class file-->instrument

Mode 3: Direct modification through ASM or other components that manipulate byte codes. class file -->instrument
...

Either way, the process is too complex and error prone. Are there mature components? Yes, Arthas and Btrace

6. Arthas&btrace

BTrace is based on Dynamic Byte Code Modification (Instrumentation) technology to track and replace Java programs at runtime. The general principle can be described in the following formula: Client(Java compile api + attach api) + Agent (script parsing engine + ASM + JDK6 Instumentation) + Socket BTrace is actually an additional agent using java attach api. Jar, then rewrite the byte code of the specified class using the script parsing engine + asm, then use instrument to replace the original class.

But BTrace scripts have a learning cost in use. It would be great if you could encapsulate some of the commonly used functions and provide simple commands to operate directly. In September 2018, Ali opened its own Java diagnostic tool, Arthas. Arthas is a powerful tool that can be accomplished through simple command line operations. The underlying technical principles are roughly the same as those described in this paper.

Btrace open source address: https://github.com/btraceio/btrace

Arthas Open Source Address: https://github.com/alibaba/arth

7. Summary

Java instruments play an important role in many applications, such as:

• apm:(Application Performance Management) Apply performance management. pinpoint, cat, skywalking, etc. are all implemented based on Instrumentation
• Hot deployment tools such as Hot Swap and Jrebel for idea
• Application Level Failure Walkthrough
• Java diagnostic tools Arthas, Btrace, and so on