Btrace java online troubleshooting artifact

What is BTrace

BTrace is a killer for checking and solving online problems. BTrace can obtain all information during program execution by writing scripts. Note that there is no need to restart the service. Yes, there is no need to restart the service. Write a script and execute it directly with commands without touching the code of the original program.


In general, BTrace is based on dynamic bytecode modification technology (Hotswap) to realize the tracking and replacement of java programs at run time. The general principle can be described by the following formula: Client(Java compile api + attach api) + Agent (script parsing engine + asm + JDK6 instrumentation) + Socket. In fact, BTrace uses java attach api to attach agent.jar, then uses script parsing engine + asm to rewrite the bytecode of the specified class, and then uses instrument to replace the original class.

install and configure

This installation and configuration is performed under Linux Ubuntu 14.04. At present, the latest version of btrace is 1.3.9, and the code is hosted on [github]. The first step is in github   Download btrace-bin-1.3.9.tgz in the releases version. There is no build directory in the zip version. Step 2: unzip btrace-bin-1.3.9.tgz to a directory, such as  / home/fengzheng/soft/btrace  , In fact, it can be used at this step, but when executing the script, you need to add an absolute path before the btrace command. If you want to be executable in any directory, go to the next step. The third step is to configure the environment variables. The configured environment variables include   JAVA_HOME and   BTRACE_HOME  , For example, my configuration is as follows:

export JAVA_HOME=/home/fengzheng/soft/jdk1.8.0_111export JRE_HOME=${JAVA_HOME}/jreexport CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib export PATH=${JAVA_HOME}/bin:$PATHexport BTRACE_HOME=/home/fengzheng/soft/btraceexport PATH=$PATH:$BTRACE_HOME/bin

Then execute the command   source /etc/profile  , Make the environment variable effective immediately. Next, execute in any directory   btrace command can be executed successfully.

Simple test case

btrace's simplest syntax is   btrace $pid, so you need to know what to probe   The process id of the Java program, and then write a probe script.

1. Write a resident memory   Java   An infinite loop is written here, and a set of calculation results are output every 5 seconds. The contents are as follows:

package kite.lab.utils; /** * NumberUtil * * @author fengzheng * @date 2017/2/15 */public class NumberUtil {     public int sum(){        int result = 0;        for(int i = 0; i< 100; i++){            result += i * i;        }        return result;    }     public static void main(String[] args){        while (true) {            Thread.currentThread().setName("calculation");            NumberUtil util = new NumberUtil();            int result = util.sum();            System.out.println(result);            try {                Thread.sleep(5000);            }catch (InterruptedException e){             }        }    }}

By the way, the process of compiling and running Java from the command line:

Compile: javac - D., locate the directory where is located, and then execute this command line. The directory structure shown in the package name, kit / Lab / utils / numberutil.class, will be generated in the current directory (. Represents the current directory)

Execution: Java kit.lab.utils.numberutil   Just

two   After executing the above procedure, it is available   jps   Command to check the PID (generally, the program started with which account should use which account to execute JPS, except the root account). Execute the JPS command and see the following results:

root@ubuntu:/home/fengzheng/codes/btrace# jps10906 Jps10860 NumberUtil

3. You can see that the java process just executed is 10860

4. Write btrace script, which is as follows:

package kite; import com.sun.btrace.annotations.*; import static com.sun.btrace.BTraceUtils.Strings.strcat;import static com.sun.btrace.BTraceUtils.jstack;import static com.sun.btrace.BTraceUtils.println;import static com.sun.btrace.BTraceUtils.str; /** * NumberUtilBTrace * * @author fengzheng * @date 2017/6/20 */@BTracepublic class NumberUtilBTrace {     @OnMethod(            clazz="kite.lab.utils.NumberUtil",            method="sum",            location=@Location(Kind.RETURN)    )    public static void func(@Return int result) {        println("trace: =======================");        println(strcat("result:", str(result)));        jstack();    }}

It means that after execution (location=@Location(Kind.RETURN) indicates the end of execution), the result and stack information are output

5. Precompiling: before execution, you can check the correctness of the script with the precompiled command. The precompiled command is btracec, which is a javac like command, btracec

six   Call the command line to execute, btrace 10860  , (if you want to save to a local file, you can use the turn command btrace 10860 numberutilbtrace. Java > mylog. Log) the printed information is as follows

trace: =======================result:328350kite.lab.utils.NumberUtil.sum(

seven   Press ctrl + c to give an exit prompt, and then press 1 to exit

Usage scenario

BTrace is an after the fact tool. The so-called after the fact tool is used when the service has been launched, but the following problems are found.

  1. For example, which methods are too slow to execute, such as monitoring methods whose execution time exceeds 1s

  2. See which methods call System.gc(), and what the call stack is like

  3. View method parameters or object properties

  4. Which methods have exceptions

More to say, in order to better solve the problem, it is better to cooperate with prior preparation and ongoing monitoring. Prior preparation is to bury the point. Log output is carried out in some methods that may cause problems. Ongoing monitoring is to use some real-time monitoring tools, such as tools with interfaces such as VisualVM and jmc or command-line tools provided by jdk, A little more advanced is to use Metrics tools such as Graphite to display it with the web interface.

Use restrictions

In order to ensure that the trace statement is read-only and minimize the impact on the detected program, BTrace has some restrictions on the trace script (for example, it cannot change the state in the traced code)

BTrace class cannot create new classes, new arrays, throw exceptions and catch exceptions,

  • Instance methods and static methods (except com.sun.btrace.BTraceUtils) cannot be called

  • Object programs and objects cannot be assigned to instances and static field s of BTrace

  • Cannot define external, internal, anonymous, local classes

  • Cannot have synchronization blocks and methods

  • There can be no cycles

  • Cannot implement interface, cannot extend class

  • You cannot use an assert statement or a class literal

Intercept method definition

@OnMethod can specify clazz, method and location. This constitutes a method / method decision under a certain class / class (clazz decision) at what time (location decision).

How to locate:

  1. Precise positioning

Directly locate a method under a class. The above test example is

  2. Regular expression positioning

The regular expression is between two "/". For example, in the following example, monitor all methods under the javax.swing package. Note that in the formal environment, the range should be as small as possible. If it is too large, the performance will be affected.

@OnMethod(clazz="/javax\\.swing\\..*/", method="/.*/")
public static void swingMethods( @ProbeClassName String probeClass, @ProbeMethodName String probeMethod) {
   print("entered " + probeClass + "."  + probeMethod);

By injecting @ probeclassname string probeclass and @ probemethodname string probemethod parameters into the definition of the interception function, tell the script the actual matching class and method name.

three   Locate by interface or inherited class

For example, to match an interface or base class that inherits or implements com.kit.base, just add a + sign in front of the class, such as

@OnMethod(clazz="+com.kite.base", method="doSome")

4. Position according to notes

Add @ before it, for example, @ OnMethod(clazz="@javax.jws.WebService", method="@javax.jws.WebMethod")

Interception time:

The interception timing is determined by the location. Of course, multiple interception timing can also be added to the same location, that is, it can be intercepted when entering the method, when returning the method, and when throwing an exception

1. Kind.Entry and Kind.Return

Indicates the start and return of the function respectively. Without writing the location, it defaults to Kind.Entry. Only get the parameter value. You can use Kind.Entry. To get the return value or execution time, you need to use Kind.Return

2. Kind.Error, Kind.Throw and Kind.Catch

Indicates that the exception is thrown, the exception is caught, and the exception occurs but is not caught. Inject a Throwable parameter into the parameter definition of the interception function to represent the exception

@OnMethod(clazz = "com.kite.demo", location = @Location(value = Kind.LINE, line = 20))
public static void onBind() {
   println("Go to line 20");
@OnMethod(clazz = "", method = "bind", location =@Location(Kind.ERROR)) 
public static void onBind(Throwable exception, @Duration long duration){ 


3. Kind.Call and Kind.Line

Kind.Call indicates which other methods are called by the monitored method, for example:

@OnMethod(clazz = "com.kite",
            method = "login",
            location = @Location(value = Kind.CALL, clazz = "/.*/", method = "/.*/", where = Where.AFTER))
    public static void onBind(@Self Object self, @TargetInstance Object instance, @TargetMethodOrField String method, @Duration long duration){
        println(strcat("self: ", str(self)));
        println(strcat("instance: ", str(instance)));
        println(strcat("method: ", str(method)));
        println(strcat("duration(ms): ", str(duration / 1000000)));

@Self represents the class of the currently monitored function, if it is a static class, it is empty. @TargetInstance indicates the class of the method or attribute that is invoked in the function. If it is static, it is empty. @TargetMethodOrField represents the method or attribute of the call. If you want to get the execution time, then where must be set to Where.AFTER.

Kind.Line monitors whether the class executes the set number of lines, for example:

@OnMethod(clazz = "com.kite.demo", location = @Location(value = Kind.LINE, line = 20))
public static void onBind() {
   println("Go to line 20");

Several examples

See who called GC

@OnMethod(clazz = "java.lang.System", method = "gc")    public static void onSystemGC() {        println("entered System.gc()");        jstack();    }

Printing methods that take more than 100ms

@OnMethod(clazz = "/com\\.kite\\.controller\\..*/",method = "/.*/",location = @Location(Kind.RETURN))
    public static void slowQuery(@ProbeClassName String pcn,@ProbeMethodName String probeMethod, @Duration long duration){
        if(duration > 1000000 * 100){
            println(strcat("Class:", pcn));
            println(strcat("method:", probeMethod));
            println(strcat("Duration:", str(duration / 1000000)));

Attention problem

If present   Unable to open socket file: target process not responding or HotSpot VM not loaded   The possible reason for this problem is that the user executing the BTrace script and the user running the Java process are not the same, so the   ps -aux | grep $pid check the execution user of the Java process and ensure that it is the same as the BTrace script execution user

Pay attention to my official account and get more knowledge about the back end and big data.


Article from: 

Keywords: Java jvm Back-end

Added by dagon on Thu, 04 Nov 2021 18:48:23 +0200