Java Record serialization related

At the beginning of the design, Record is to find a type carrier that purely represents data. Java's class is now doing function addition through continuous iteration, and its usage has been very complex. Various syntax sugars, various polymorphic constructors and various inheritance designs lead to the very complex serialization framework for Java. There are many situations to consider. Every time Java is upgraded, if the class structure is changed or new features are added, the serialization framework needs to be changed to be compatible. This will hinder the development of Java, so Record is designed to store data.

Through the analysis in the previous section, we know that the Record type is final after declaration. After compilation, insert the bytecode of relevant fields and methods according to the Record source code, including:

1. Automatically generated private final field
2. Auto generated full attribute constructor
3. Auto generated public getter method
4. Automatically generated hashCode(), equals(), toString() method:
5. As can be seen from the bytecode, the underlying implementation of these three methods is another invokeDynamic method
6. Objectmethods. Is called bootstrap method in Java class

All elements are immutable, which is much more convenient for serialization and omits many factors to be considered, such as field parent-child class inheritance and coverage. To serialize a Record, you only need to pay attention to the Record itself and read out all the fields in it, and these fields are final. During deserialization, only the canonical constructor of Record, that is, the constructor that assigns values to all attributes.

Next, let's take a simple example to see the difference between Record and ordinary class serialization.

Here we use lombok to simplify the code, assuming that there is UserClass:

@Data
public class UserClass implements Serializable {
	private final int id;
	private final int age;
}
Copy code

There is also a UserRecord with the same field as it:

public record UserRecord(int id, int age) implements Serializable {}
Copy code

Write code that uses Java Native serialization:

public class SerializationTest {
	public static void main(String[] args) throws Exception {
		try (FileOutputStream fileOutputStream = new  FileOutputStream("data");
			 ObjectOutputStream objectOutputStream = new ObjectOutputStream(fileOutputStream)) {
			//Write UserClass first
			objectOutputStream.writeObject(new UserClass(1, -1));
			//Write UserRecord again
			objectOutputStream.writeObject(new UserRecord(2, -1));
		}
	}
}
Copy code

Execute, write the two objects into the file data, and then write code to read from this file and output:

public class DeSerializationTest {
	public static void main(String[] args) throws Exception {
		try (FileInputStream fileInputStream = new  FileInputStream("data");
			 ObjectInputStream objectInputStream = new ObjectInputStream(fileInputStream)) {
			//Read UserClass
			System.out.println(objectInputStream.readObject());
			//Read UserRecord
			System.out.println(objectInputStream.readObject());
		}
	}
}
Copy code

After execution, you will see the output:

UserClass(id=1, age=-1)
UserRecord[id=1, age=-1]
Copy code

Constructor test

Next, we modify the source code and add the judgment that neither id nor age can be less than 1 in UserClass and UserRecord. In addition, an additional constructor is added to UserRecord to verify that the full attribute constructor of UserRecord is used for deserialization.

@Data
public class UserClass implements Serializable {
	private final int id;
	private final int age;

	public UserClass(int id, int age) {
		if (id < 0 || age < 0) {
			throw new IllegalArgumentException("id and age should be larger than 0");
		}
		this.id = id;
		this.age = age;
	}
}
public record UserRecord(int id, int age) implements Serializable {
	public UserRecord {
		if (id < 0 || age < 0) {
			throw new IllegalArgumentException("id and age should be larger than 0");
		}
	}

	public UserRecord(int id) {
		this(id, 0);
	}
}
Copy code

Execute the code DeSerializationTest again. We will find that there is an error, but the UserClass is deserialized:

UserClass(id=1, age=-1)
Exception in thread "main" java.io.InvalidObjectException: id and age should be larger than 0
	at java.base/java.io.ObjectInputStream.readRecord(ObjectInputStream.java:2348)
	at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2236)
	at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1742)
	at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:514)
	at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:472)
	at DeSerializationTest.main(DeSerializationTest.java:13)
Caused by: java.lang.IllegalArgumentException: id and age should be larger than 0
	at UserRecord.<init>(UserRecord.java:6)
	at java.base/java.io.ObjectInputStream.readRecord(ObjectInputStream.java:2346)
	... 5 more

Copy code

Compatibility test

Let's look at what happens if you delete a field:

@Data
public class UserClass implements Serializable {
	private final int age;
}
public record UserRecord(int age) implements Serializable {
}
Copy code

When executing the code, an error will be reported when reading UserClass, which is also expected, because it is said in the deserialization description of ordinary class objects that this is an incompatible modification. Restore the field of UserClass and re execute the code. The discovery is successful:

UserClass(id=1, age=-1)
UserRecord[age=-1]
Copy code

That is, Record is compatible with deserialization of missing fields by default

Let's restore the field and see what happens to more than one field:

@Data
public class UserClass implements Serializable {
	private final int id;
	private final int sex;
	private final int age;
}
public record UserRecord(int id, int sex, int age) implements Serializable {
}
Copy code

Execute the code and an error will be reported when reading UserClass, which is also expected. Restore the field of UserClass and re execute the code. The discovery is successful:

UserClass(id=1, age=-1)
UserRecord[id=2, sex=0, age=-1]
Copy code

In other words, Record is deserialized with more compatible fields by default

Finally, test the field type of Record. What if it changes

public record UserRecord(int id, Integer age) implements Serializable {
}
Copy code

Code execution failed because the type does not match (not even the wrapper class):

UserClass(id=1, age=-1)
Exception in thread "main" java.io.InvalidClassException: UserRecord; incompatible types for field age
	at java.base/java.io.ObjectStreamClass.matchFields(ObjectStreamClass.java:2391)
	at java.base/java.io.ObjectStreamClass.getReflector(ObjectStreamClass.java:2286)
	at java.base/java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:788)
	at java.base/java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:2060)
	at java.base/java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1907)
	at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2209)
	at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1742)
	at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:514)
	at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:472)
	at DeSerializationTest.main(DeSerializationTest.java:13)
Copy code

Compatibility with some mainstream serialization frameworks

Because Record limits the only way of serialization and deserialization, it is actually very simple to be compatible, which is simpler than the serialization framework change caused by changing the structure of Java Class and adding a feature.

• Jackson:

  • Issue: Support for record types in JDK 14
  • Pull Request: Support for record types in JDK 14
  • Corresponding version: jackson-databind-2.12.0

• Kryo

  • Issue: Java 14 records : how to deal with them?
  • Pull Request: Add support for Records in JDK 14 
  • Corresponding version: kryo-5.1.0

• XStream

  • Issue: Support for record types in JDK 14
  • Pull Request: Add support for Record types in JDK 14
  • Corresponding version: 1.5 x. Not yet released

The idea of implementing Record compatibility in these three frameworks is very similar and relatively simple, that is:

1. Implement a special Serializer and Deserializer for Record.
2. Verify whether the current version of Java supports Record through Java Reflection or Java MethodHandle, and obtain the canonical constructor of Record and getter s of various field s for deserialization and serialization. For your reference, two tool classes are implemented using Java Reflection and Java MethodHandle:

import java.lang.reflect.Constructor;
import java.lang.reflect.Method;
import java.util.Arrays;
import java.util.Comparator;
import common.RecComponent;

/**
 * Utility methods for record serialization, using Java Core Reflection.
 */
public class ReflectUtils {
    private static final Method IS_RECORD;
    private static final Method GET_RECORD_COMPONENTS;
    private static final Method GET_NAME;
    private static final Method GET_TYPE;

    static {
        Method isRecord;
        Method getRecordComponents;
        Method getName;
        Method getType;

        try {
            // reflective machinery required to access the record components
            // without a static dependency on Java SE 14 APIs
            Class<?> c = Class.forName("java.lang.reflect.RecordComponent");
            isRecord = Class.class.getDeclaredMethod("isRecord");
            getRecordComponents = Class.class.getMethod("getRecordComponents");
            getName = c.getMethod("getName");
            getType = c.getMethod("getType");
        } catch (ClassNotFoundException | NoSuchMethodException e) {
            // pre-Java-14
            isRecord = null;
            getRecordComponents = null;
            getName = null;
            getType = null;
        }

        IS_RECORD = isRecord;
        GET_RECORD_COMPONENTS = getRecordComponents;
        GET_NAME = getName;
        GET_TYPE = getType;
    }

    /** Returns true if, and only if, the given class is a record class. */
    static boolean isRecord(Class<?> type) {
        try {
            return (boolean) IS_RECORD.invoke(type);
        } catch (Throwable t) {
            throw new RuntimeException("Could not determine type (" + type + ")");
        }
    }

    /**
     * Returns an ordered array of the record components for the given record
     * class. The order is imposed by the given comparator. If the given
     * comparator is null, the order is that of the record components in the
     * record attribute of the class file.
     */
    static <T> RecComponent[] recordComponents(Class<T> type,
                                               Comparator<RecComponent> comparator) {
        try {
            Object[] rawComponents = (Object[]) GET_RECORD_COMPONENTS.invoke(type);
            RecComponent[] recordComponents = new RecComponent[rawComponents.length];
            for (int i = 0; i < rawComponents.length; i++) {
                final Object comp = rawComponents[i];
                recordComponents[i] = new RecComponent(
                        (String) GET_NAME.invoke(comp),
                        (Class<?>) GET_TYPE.invoke(comp), i);
            }
            if (comparator != null) Arrays.sort(recordComponents, comparator);
            return recordComponents;
        } catch (Throwable t) {
            throw new RuntimeException("Could not retrieve record components (" + type.getName() + ")");
        }
    }

    /** Retrieves the value of the record component for the given record object. */
    static Object componentValue(Object recordObject,
                                         RecComponent recordComponent) {
        try {
            Method get = recordObject.getClass().getDeclaredMethod(recordComponent.name());
            return get.invoke(recordObject);
        } catch (Throwable t) {
            throw new RuntimeException("Could not retrieve record components ("
                    + recordObject.getClass().getName() + ")");
        }
    }

    /**
     * Invokes the canonical constructor of a record class with the
     * given argument values.
     */
    static <T> T invokeCanonicalConstructor(Class<T> recordType,
                                                    RecComponent[] recordComponents,
                                                    Object[] args) {
        try {
            Class<?>[] paramTypes = Arrays.stream(recordComponents)
                    .map(RecComponent::type)
                    .toArray(Class<?>[]::new);
            Constructor<T> canonicalConstructor = recordType.getConstructor(paramTypes);
            return canonicalConstructor.newInstance(args);
        } catch (Throwable t) {
            throw new RuntimeException("Could not construct type (" + recordType.getName() + ")");
        }
    }
}
Copy code

package invoke;

import common.RecComponent;
import java.lang.invoke.MethodHandle;
import java.lang.invoke.MethodHandles;
import java.lang.reflect.Array;
import java.util.Arrays;
import java.util.Comparator;
import static java.lang.invoke.MethodType.methodType;

/**
 * Utility methods for record serialization, using MethodHandles.
 */
public class InvokeUtils {
    private static final MethodHandle MH_IS_RECORD;
    private static final MethodHandle MH_GET_RECORD_COMPONENTS;
    private static final MethodHandle MH_GET_NAME;
    private static final MethodHandle MH_GET_TYPE;
    private static final MethodHandles.Lookup LOOKUP;

    static {
        MethodHandle MH_isRecord;
        MethodHandle MH_getRecordComponents;
        MethodHandle MH_getName;
        MethodHandle MH_getType;
        LOOKUP = MethodHandles.lookup();

        try {
            // reflective machinery required to access the record components
            // without a static dependency on Java SE 14 APIs
            Class<?> c = Class.forName("java.lang.reflect.RecordComponent");
            MH_isRecord = LOOKUP.findVirtual(Class.class, "isRecord", methodType(boolean.class));
            MH_getRecordComponents = LOOKUP.findVirtual(Class.class, "getRecordComponents",
                    methodType(Array.newInstance(c, 0).getClass()))
                    .asType(methodType(Object[].class, Class.class));
            MH_getName = LOOKUP.findVirtual(c, "getName", methodType(String.class))
                    .asType(methodType(String.class, Object.class));
            MH_getType = LOOKUP.findVirtual(c, "getType", methodType(Class.class))
                    .asType(methodType(Class.class, Object.class));
        } catch (ClassNotFoundException | NoSuchMethodException e) {
            // pre-Java-14
            MH_isRecord = null;
            MH_getRecordComponents = null;
            MH_getName = null;
            MH_getType = null;
        } catch (IllegalAccessException unexpected) {
            throw new AssertionError(unexpected);
        }

        MH_IS_RECORD = MH_isRecord;
        MH_GET_RECORD_COMPONENTS = MH_getRecordComponents;
        MH_GET_NAME = MH_getName;
        MH_GET_TYPE = MH_getType;
    }

    /** Returns true if, and only if, the given class is a record class. */
    static boolean isRecord(Class<?> type) {
        try {
            return (boolean) MH_IS_RECORD.invokeExact(type);
        } catch (Throwable t) {
            throw new RuntimeException("Could not determine type (" + type + ")");
        }
    }

    /**
     * Returns an ordered array of the record components for the given record
     * class. The order is imposed by the given comparator. If the given
     * comparator is null, the order is that of the record components in the
     * record attribute of the class file.
     */
    static <T> RecComponent[] recordComponents(Class<T> type,
                                               Comparator<RecComponent> comparator) {
        try {
            Object[] rawComponents = (Object[]) MH_GET_RECORD_COMPONENTS.invokeExact(type);
            RecComponent[] recordComponents = new RecComponent[rawComponents.length];
            for (int i = 0; i < rawComponents.length; i++) {
                final Object comp = rawComponents[i];
                recordComponents[i] = new RecComponent(
                        (String) MH_GET_NAME.invokeExact(comp),
                        (Class<?>) MH_GET_TYPE.invokeExact(comp), i);
            }
            if (comparator != null) Arrays.sort(recordComponents, comparator);
            return recordComponents;
        } catch (Throwable t) {
            throw new RuntimeException("Could not retrieve record components (" + type.getName() + ")");
        }
    }

    /** Retrieves the value of the record component for the given record object. */
    static Object componentValue(Object recordObject,
                                         RecComponent recordComponent) {
        try {
            MethodHandle MH_get = LOOKUP.findVirtual(recordObject.getClass(),
                    recordComponent.name(),
                    methodType(recordComponent.type()));
            return (Object) MH_get.invoke(recordObject);
        } catch (Throwable t) {
            throw new RuntimeException("Could not retrieve record components ("
                    + recordObject.getClass().getName() + ")");
        }
    }

    /**
     * Invokes the canonical constructor of a record class with the
     * given argument values.
     */
    static <T> T invokeCanonicalConstructor(Class<T> recordType,
                                                    RecComponent[] recordComponents,
                                                    Object[] args) {
        try {
            Class<?>[] paramTypes = Arrays.stream(recordComponents)
                    .map(RecComponent::type)
                    .toArray(Class<?>[]::new);
            MethodHandle MH_canonicalConstructor =
                    LOOKUP.findConstructor(recordType, methodType(void.class, paramTypes))
                            .asType(methodType(Object.class, paramTypes));
            return (T)MH_canonicalConstructor.invokeWithArguments(args);
        } catch (Throwable t) {
            throw new RuntimeException("Could not construct type (" + recordType.getName() + ")");
        }
    }
}