Original title: Spring certified China Education Management Center - Spring Data Redis framework tutorial I
8.1. Document structure
This section of the reference document explains the core functions provided by Spring Data Redis. It explains the concept and semantics of key value modules and the syntax of various store namespaces. For an introduction to key value storage, Spring or Spring Data examples, see learning NoSQL and key value storage. This document only covers Spring Data Redis support and assumes that users are familiar with key value storage and Spring concepts.
"Redis support" describes the redis module function set.
"Redis Repositories" describes the repository support for redis.
This document is a reference guide for Spring Data Redis (SDR) support.
9. Why use Spring Data Redis?
The Spring Framework is the leading full stack Java/JEE application framework. It provides a lightweight container and a non intrusive programming model by using dependency injection, AOP and portable service abstraction.
NoSQL storage system provides an alternative to the classic RDBMS to achieve horizontal scalability and speed. In terms of implementation, the key value store represents one of the largest (and oldest) members in the NoSQL space.
Spring Data Redis (SDR) framework eliminates redundant tasks and template code required for interaction with storage through spring's excellent infrastructure support, so that spring applications using Redis key value storage can be easily written.
10. Redis support
One of the key value stores supported by Spring Data is Redis. Refer to Redis project homepage:
Redis is an advanced key value storage. It is similar to memcached, but the data set is not volatile. The value can be a string, just as in memcached, but it can also be a list, set and ordered set. All these data types can be operated through atomic operations to push / pop elements, add / delete elements, perform server-side Union, intersection, differences between collections, etc. Redis supports different sort capabilities.
Spring Data Redis provides the ability to easily configure and access Redis from spring applications. It provides low-level and high-level abstractions for interacting with stores, freeing users from infrastructure concerns.
10.1. Getting started
A simple way to set up the working environment is to create a Spring based project in STS.
First, you need to set up a running Redis server.
Create a Spring project in STS:
- Go to file → new → spring template project → simple spring utility project, and press Yes when prompted. Then enter the project and package name, such as org.spring.redis.example. Add the following to the dependencies element of the pom.xml file: < dependencies > <-- other dependency elements omitted --> <dependency> <groupId>org.springframework.data</groupId> <artifactId>spring-data-redis</artifactId> <version>2.6.0</version> </dependency> </dependencies>
- Change the Spring version in pom.xml to < Spring. Framework. Version > 5.3.13 < / Spring. Framework. Version >
- Add the following location of Maven's Spring Milestone repository to the < dependencies / > element of your pom.xml so that it is at the same level as your element: < repositories > < repository > < ID > Spring Milestone < / ID > < name > spring Maven milestone repository < / name > < URL > https://repo.spring.io/libs-milestone </url> </repository> </repositories>
The repository can also be browsed here.
10.2.Redis requirements
Spring Redis requires Redis 2.6 or higher, and Spring Data Redis is integrated with lattice and Jedis, two popular Redis open source Java libraries.
10.3.Redis supports advanced views
Redis supports several components. For most tasks, high-level abstraction and support services are the best choice. Note that you can move between layers at any time. For example, you can get a low-level connection (even a native library) to communicate directly with redis.
10.4. Connect to Redis
One of the first tasks when using Redis and Spring is to connect to the storage through the IoC container. To do this, you need a Java connector (or binding). No matter the library you choose, you only need to use a set of Spring Redis data API (which has the same behavior in all connectors): in The org.springframework.data.redis.connection package and its RedisConnection and RedisConnectionFactory are connected to the working and retrieved Redis activities.
10.4.1.RedisConnection and RedisConnectionFactory
RedisConnection provides the core building block for Redis communication because it handles the communication with the Redis backend. It also automatically converts the underlying connection library exceptions into a Spring consistent DAO exception hierarchy so that you can switch connectors without changing any code, because the operation semantics remain the same.
For extreme cases that require native library API s, RedisConnection provides a special method (getNativeConnection), which returns the original underlying object used for communication.
The active RedisConnection object is through RedisConnectionFactory. In addition, the factory acts as PersistenceExceptionTranslator objects, which means that once declared, they allow you to perform transparent exception conversion. For example, you can translate exceptions by using @ Repository annotations and AOP. For more information, see the dedicated section in the Spring Framework documentation.
Depending on the underlying configuration, the factory can return new or existing connections (when using pooled or shared native connections).
The easiest way to use a RedisConnectionFactory is to configure the appropriate connector through the IoC container and inject it into the using class.
Unfortunately, not all connectors currently support all Redis functions. When a method is called on a Connection API that is not supported by the underlying library, an unsupported operationexception will be thrown. The following overview describes the functions supported by each Redis connector:
10.4.2. Configure lettuce connector
Lattice is a Netty based open source connector supported by Spring Data Redis package org.springframework.data.redis.connection.lettuce.
Add the following to the dependencies element of the pom.xml file:
<dependencies>
<!-- other dependency elements omitted -->
<dependency>
<groupId>io.lettuce</groupId>
<artifactId>lettuce-core</artifactId>
<version>6.1.5.RELEASE</version>
</dependency>
</dependencies>The following example shows how to create a new lattice connection factory:
@Configuration
class AppConfig {
@Bean
public LettuceConnectionFactory redisConnectionFactory() {
return new LettuceConnectionFactory(new RedisStandaloneConfiguration("server", 6379));
}
}There are also some lettuce specific connection parameters that can be adjusted. By default, letticeconnection consists of all instances created. Letticeconnectionfactory shares the same thread safe native connection for all non blocking and non transactional operations. To use a private connection each time, set shareNativeConnection to false. If set to, letticeconnectionfactory can also be configured to use aLettucePool for pooling blocking and transactional connections or all connections. shareNativeConnectionfalse
Lettue's native transport integration with Netty allows you to communicate with Redis using Unix domain sockets. Ensure that you include the appropriate native transport dependencies that match your runtime environment. The following example shows how to create a Lettuce connection factory / var/run/redis.sock for a Unix domain socket:
@Configuration
class AppConfig {
@Bean
public LettuceConnectionFactory redisConnectionFactory() {
return new LettuceConnectionFactory(new RedisSocketConfiguration("/var/run/redis.sock"));
}
}Netty currently supports epoll (Linux) and kqueue (BSD/macOS) interfaces for operating system native transport. 10.4.3. Configure Jedis connector
Jedis is the Spring Data Redis module The org.springframework.data.redis.connection.jedis package supports community driven connectors.
Add the following to the dependencies element of the pom.xml file:
<dependencies>
<!-- other dependency elements omitted -->
<dependency>
<groupId>redis.clients</groupId>
<artifactId>jedis</artifactId>
<version>3.7.0</version>
</dependency>
</dependencies>The simplest form of Jedis configuration is as follows:
@Configuration
class AppConfig {
@Bean
public JedisConnectionFactory redisConnectionFactory() {
return new JedisConnectionFactory();
}
}However, for production purposes, you may need to adjust settings such as host or password, as shown in the following example:
@Configuration
class RedisConfiguration {
@Bean
public JedisConnectionFactory redisConnectionFactory() {
RedisStandaloneConfiguration config = new RedisStandaloneConfiguration("server", 6379);
return new JedisConnectionFactory(config);
}
}10.4.4. Write to the master server and read from the replica
Redis master / replica settings - no automatic failover (for automatic failover, see sentry) - not only allow data to be stored securely on more nodes. It also allows data to be read from the replica by using lettue while pushing writes to the primary server. You can use to set the read / write policy to be used, Lettueclientconfiguration is shown in the following example:
@Configuration
class WriteToMasterReadFromReplicaConfiguration {
@Bean
public LettuceConnectionFactory redisConnectionFactory() {
LettuceClientConfiguration clientConfig = LettuceClientConfiguration.builder()
.readFrom(REPLICA_PREFERRED)
.build();
RedisStandaloneConfiguration serverConfig = new RedisStandaloneConfiguration("server", 6379);
return new LettuceConnectionFactory(serverConfig, clientConfig);
}
}For environments where non-public addresses are reported through the INFO command (for example, when using AWS), use the RedisStaticMasterReplicaConfiguration replaces RedisStandaloneConfiguration. Note that RedisStaticMasterReplicaConfiguration does not support Pub/Sub because it lacks Pub/Sub message propagation across a single server.
10.5.Redis sentinel support
For handling highly available Redis, Spring Data Redis already supports Redis Sentinel, using RedisSentinelConfiguration, as shown in the following example:
/**
* Jedis
*/
@Bean
public RedisConnectionFactory jedisConnectionFactory() {
RedisSentinelConfiguration sentinelConfig = new RedisSentinelConfiguration()
.master("mymaster")
.sentinel("127.0.0.1", 26379)
.sentinel("127.0.0.1", 26380);
return new JedisConnectionFactory(sentinelConfig);
}
/**
* Lettuce
*/
@Bean
public RedisConnectionFactory lettuceConnectionFactory() {
RedisSentinelConfiguration sentinelConfig = new RedisSentinelConfiguration()
.master("mymaster")
.sentinel("127.0.0.1", 26379)
.sentinel("127.0.0.1", 26380);
return new LettuceConnectionFactory(sentinelConfig);
}
RedisSentinelConfiguration can also define PropertySource with a, which allows you to set the following properties:
Configuration properties
spring.redis.sentinel.master: the name of the master node.
spring.redis.sentinel.nodes: comma separated host: port pair list.
spring.redis.sentinel.password: the password applied when using Redis Sentinel for authentication
Sometimes, you need to interact directly with one of the sentinels. use RedisConnectionFactory.getSentinelConnection() or RedisConnection.getSentinelCommands() allow you to access the first active Sentinel configured.
10.6. Processing objects through RedisTemplate
Most users may use RedisTemplate and its corresponding package, org.springframework.data.redis.core. Due to its rich function set, the template is actually the central class of Redis module. This template provides high-level abstraction for Redis interaction. Although RedisConnection provides low-level methods to accept and return binary values (byte arrays), the template is responsible for serialization and connection management, freeing users from handling such details.
In addition, the template provides an operation view (following the grouping in the Redis Command Reference) and provides rich general interfaces for working for specific types or specific keys (through the KeyBound interface), as described in the following table:
After configuration, the template is thread safe and can be reused across multiple instances.
RedisTemplate uses Java based serializers for most operations. This means that any object written or read by the template is serialized and deserialized through Java. You can change the serialization mechanism on the template. Redis module provides a variety of implementations, which can be found in org.springframework.data.redis.serializer package. For more information, see serializer. You can also set any serializer to null and set the enableDefaultSerializer property to RedisTemplate and the original byte array to false. Note that the template requires all keys to be non empty. However, as long as the underlying serializer accepts them, the value can be empty. Read the Javadoc for each serializer for more information.
Where a template view is required, declare the view as a dependency and inject the template. The container automatically performs the conversion and eliminates opsFor[X] calls, as shown in the following example:
<?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:p="http://www.springframework.org/schema/p" xsi:schemaLocation="http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans.xsd"> <bean id="jedisConnectionFactory" class="org.springframework.data.redis.connection.jedis.JedisConnectionFactory" p:use-pool="true"/> <!-- redis template definition --> <bean id="redisTemplate" class="org.springframework.data.redis.core.RedisTemplate" p:connection-factory-ref="jedisConnectionFactory"/> ... </beans>
public class Example {
// inject the actual template
@Autowired
private RedisTemplate<String, String> template;
// inject the template as ListOperations
@Resource(name="redisTemplate")
private ListOperations<String, String> listOps;
public void addLink(String userId, URL url) {
listOps.leftPush(userId, url.toExternalForm());
}
}10.7. String centered convenience courses
Because the keys and values stored in Redis are common java.lang.String, the Redis module provides two extensions of RedisConnection and RedisTemplate, StringRedisConnection (and its DefaultStringRedisConnection Implementation) and StringRedisTemplate are convenient one-stop solutions for intensive String operations. In addition to binding to String keys, templates and connections use the underlying StringRedisSerializer, which means that the stored keys and values are human readable (assuming the same encoding is used in Redis and your code). The following listing shows an example:
<?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:p="http://www.springframework.org/schema/p" xsi:schemaLocation="http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans.xsd"> <bean id="jedisConnectionFactory" class="org.springframework.data.redis.connection.jedis.JedisConnectionFactory" p:use-pool="true"/> <bean id="stringRedisTemplate" class="org.springframework.data.redis.core.StringRedisTemplate" p:connection-factory-ref="jedisConnectionFactory"/> ... </beans>
public class Example {
@Autowired
private StringRedisTemplate redisTemplate;
public void addLink(String userId, URL url) {
redisTemplate.opsForList().leftPush(userId, url.toExternalForm());
}
}As with other Spring templates, RedisTemplate and StringRedisTemplate let you talk directly through RedisCallback interface of Redis. This function gives you complete control because it is directly connected with RedisConnection. Please note that the callback receives the instance StringRedisTemplate when using stringredisconnection a. The following example shows how to use the RedisCallback interface:
public void useCallback() {
redisTemplate.execute(new RedisCallback<Object>() {
public Object doInRedis(RedisConnection connection) throws DataAccessException {
Long size = connection.dbSize();
// Can cast to StringRedisConnection if using a StringRedisTemplate
((StringRedisConnection)connection).set("key", "value");
}
});
}10.8. Serializer
From the perspective of the framework, the data stored in Redis is only bytes. Although Redis itself supports various types, in most cases, these types refer to the storage method of data rather than the content it represents. It is up to the user to decide whether to convert the information to a string or any other object.
In Spring Data, the conversion between user (custom) types and raw data (and vice versa) is The org.springframework.data.redis.serializer package is handled by Redis.
This package contains two types of serializers. As the name suggests, they are responsible for the serialization process:
- Based on RedisSerializer
- The element reader and writer using RedisElementReader and RedisElementWriter.
The main difference between these variants is that RedisSerializer mainly uses byte [] in reader and author
Several implementations are available (including the two already mentioned in this document):
- JdkSerializationRedisSerializer is used for RedisCache and RedisTemplate by default.
- StringRedisSerializer for.
However, OxmSerializer can be used for object / XML Mapping through Spring OXM support, Jackson2JsonRedisSerializer or GenericJackson2JsonRedisSerializer is used to store data in JSON format.
Note that the storage format is not limited to values. It can be used for keys, values, or hashes without any restrictions.
By default, RedisCache and RedisTemplate are configured to use Java Native serialization. Java Native serialization is known for allowing remote code to run caused by payloads that inject unauthenticated bytecode using vulnerable libraries and classes. Manipulating input can cause unwanted code to run in the application during the deserialization step. Therefore, do not use serialization in an untrusted environment. In general, we strongly recommend that you use any other message format (such as JSON).
If you are worried about security vulnerabilities caused by Java serialization, please consider the common serialization filter mechanism at the core JVM level, which was originally developed for JDK 9, but later transplanted to JDK 8, 7 and 6:
Filter incoming serialized data.
JEP 290.
OWASP: deserialization of untrusted data.
10.9. Hash mapping
Various data structures in Redis can be used to store data. Jackson2JsonRedisSerializer can convert JSON formatted objects. Ideally, JSON can be stored as a value using normal keys. You can implement more complex structured object mapping by using Redis hash. Spring Data Redis provides various strategies for mapping data to hash (depending on the use case):
- Direct mapping by using HashOperations and serializer
- Using Redis repository
- Using HashMapper and HashOperations
10.9.1. Hash mapper
The hash mapper is a converter that maps objects to amap < K, V > and returns. HashMapper is intended for use with Redis hashes.
Several implementations are available:
- Bean utils hashmapper uses Spring's bean utils.
- ObjectHashMapper uses object to hash mapping.
- Jackson2HashMapper uses fasterxml.
The following example shows a way to implement hash mapping:
public class Person {
String firstname;
String lastname;
// ...
}
public class HashMapping {
@Autowired
HashOperations<String, byte[], byte[]> hashOperations;
HashMapper<Object, byte[], byte[]> mapper = new ObjectHashMapper();
public void writeHash(String key, Person person) {
Map<byte[], byte[]> mappedHash = mapper.toHash(person);
hashOperations.putAll(key, mappedHash);
}
public Person loadHash(String key) {
Map<byte[], byte[]> loadedHash = hashOperations.entries("key");
return (Person) mapper.fromHash(loadedHash);
}
}
10.9.2.Jackson2HashMapper
Jackson2HashMapper uses fasterxml to provide Redis hash mapping for domain objects. Jackson2HashMapper can map top-level attributes to hash field names and optionally flatten the structure. Simple types map to simple values. Complex types (nested objects, collections, mappings, etc.) are represented as nested JSON.
Flatten creates separate hash entries for all nested attributes and resolves complex types to simple types as much as possible.
Consider the following classes and the data structures they contain:
public class Person {
String firstname;
String lastname;
Address address;
Date date;
LocalDateTime localDateTime;
}
public class Address {
String city;
String country;
}The following table shows how the data in the previous class is displayed in the normal mapping:
Flattening requires that all attribute names do not interfere with JSON paths. Using flattening, points or parentheses in mapping keys or as attribute names are not supported. The generated hash cannot be mapped back to the object.
java.util.Date and java.util.Calendar are expressed in milliseconds. If the JSR-310 date / time type is on the classpath, toString serializes it to its form jackson-datatype-jsr310.
10.10.Redis messaging (publish / subscribe)
Spring Data provides Redis with special messaging integration, which is similar to JMS integration in Spring Framework in function and naming.
Redis messaging can be roughly divided into two functions:
- Release or production of news
- Subscription or consumption of messages
This is an example of a pattern commonly referred to as publish / subscribe (Pub/Sub). The RedisTemplate class is used for message generation. For message driven bean style asynchronous reception similar to Java EE, Spring Data provides a special message listener container for creating message driven POJO s (MDP s) and RedisConnection contracts for synchronous reception.
stay The org.springframework.data.redis.connection and org.springframework.data.redis.listener packages provide the core functions of Redis messages.
10.10.1. Publish (send message)
To publish a message, you can use low-level RedisConnection or high-level RedisTemplate like other operations. Both entities provide methods for publishing to accept messages and target channels as parameters. Although RedisConnection requires original data (byte array), RedisTemplate allows any object to be passed in as a message, as shown in the following example:
// send message through connection RedisConnection con = ...
byte[] msg = ...
byte[] channel = ...
con.publish(msg, channel); // send message through RedisTemplate
RedisTemplate template = ...
template.convertAndSend("hello!", "world");10.10.2. Subscribe (receive messages)
At the receiving end, one or more channels can be subscribed by direct naming or using pattern matching. The latter method is useful because it not only allows you to create multiple subscriptions with one command, but also listens for channels that were not created at the time of the subscription (as long as they match the pattern).
At the bottom layer, RedisConnection provides subscribe and pSubscribe methods to map Redis commands to subscribe by channel or mode respectively. Note that you can use multiple channels or modes as parameters. To change the subscription of a connection or query whether it is listening, RedisConnection provides getSubscription and isSubscribed methods.
The subscription command in Spring Data Redis is blocked. That is, calling subscribe on the connection will cause the current thread to block when it starts waiting for a message. The thread is released only when the subscription is cancelled, which occurs when another thread calls unsubscribe or pusubscribe on the same connection. For a solution to this problem, see "message listener container" (later in this document).
As mentioned earlier, once subscribed, the connection starts waiting for messages. Only commands that add new subscriptions, modify existing subscriptions, and cancel existing subscriptions are allowed. Calling anything more than subscribe, pSubscribe, unsubscribe, or pSubscribe throws an exception.
In order to subscribe to messages, you need to implement the MessageListener callback. Each time a new message arrives, the callback is called and the user code is run through the onMessage method. The interface can access not only the actual message, but also the channel received through it and the pattern (if any) subscribed to match the channel. This information enables the callee to distinguish messages not only by content but also by examining other details.
Message listener container
Because of its blocking nature, low-level subscriptions are unattractive because they require connection and thread management for each listener. To alleviate this problem, Spring Data provides Redismessagelistener container, which completes all the heavy work. If you are familiar with EJB and JMS, you should find these concepts familiar because it is designed to be as close as possible to the support in the Spring Framework and its message driven POJO (MDP).
RedisMessageListenerContainer acts as a message listener container. It is used to receive messages from the Redis channel and drive the MessageListener to inject instances into it. The listener container is responsible for all threads receiving messages and dispatching them to the listener for processing. The message listener container is the intermediary between MDP and the message provider. It is responsible for registering and receiving messages, obtaining and releasing resources, exception conversion, etc. This allows you, as an application developer, to write (possibly complex) business logic related to receiving messages (and responding to them), and delegate the template Redis infrastructure issues to the framework.
`MessageListener` It can also be realized `SubscriptionListener` To subscribe in/Receive notifications on unsubscribe confirmation. Listening for subscription notifications is useful when synchronous calls are made.
In addition, to minimize application footprint, RedisMessageListenerContainer allows multiple listeners to share a connection and a thread even if they do not share a subscription. Therefore, no matter how many listeners or channels the application tracks, the runtime cost remains the same throughout its life cycle. In addition, the container allows runtime configuration changes so that you can add or remove listeners while the application is running without restarting. In addition, the container uses the lazy subscription method, and RedisConnection is used only when needed. If all listeners are unsubscribed, cleanup is performed automatically and the thread is released.
To help with the asynchronous nature of messages, the container needs a java.util.concurrent.Executor (or Spring's TaskExecutor) to dispatch messages. Depending on the load, the number of listeners, or the runtime environment, you should change or adjust the executor to better meet your needs. Especially in a managed environment (such as an application server), it is strongly recommended to select an appropriate task executor to take advantage of its runtime.
MessageListenerAdapter
This MessageListenerAdapter class is the last component of Spring's asynchronous message support. In short, it allows you to expose almost any class as MDP (despite some limitations).
Consider the following interface definitions:
public interface MessageDelegate {
void handleMessage(String message);
void handleMessage(Map message); void handleMessage(byte[] message);
void handleMessage(Serializable message);
// pass the channel/pattern as well
void handleMessage(Serializable message, String channel);
}Note that although the interface does not extend the MessageListener interface, it can still be used as an MDP by using the MessageListenerAdapter class. Also note how to use various Message processing methods, which are different Message types according to strongly typed content, they can receive and process. In addition, the channel or mode to which the Message is sent can be passed to the method String as the second parameter of type:
public class DefaultMessageDelegate implements MessageDelegate {
// implementation elided for clarity...
}Note that the MessageDelegate interface implementation above (DefaultMessageDelegate class above) does not depend on Redis at all. It is indeed a POJO. We use the following configuration to make it MDP:
<?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:redis="http://www.springframework.org/schema/redis" xsi:schemaLocation="http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans.xsd http://www.springframework.org/schema/redis https://www.springframework.org/schema/redis/spring-redis.xsd"> <!-- the default ConnectionFactory --> <redis:listener-container> <!-- the method attribute can be skipped as the default method name is "handleMessage" --> <redis:listener ref="listener" method="handleMessage" topic="chatroom" /> </redis:listener-container> <bean id="listener" class="redisexample.DefaultMessageDelegate"/> ... <beans>
The listener topic can be a channel (for example, topic="chatroom") or a mode (for example, topic="*room")
The above example uses the Redis namespace to declare the message listening container and automatically register the POJO as a listener. The complete bean definition is as follows:
<bean id="messageListener" class="org.springframework.data.redis.listener.adapter.MessageListenerAdapter">
<constructor-arg>
<bean class="redisexample.DefaultMessageDelegate"/>
</constructor-arg>
</bean>
<bean id="redisContainer" class="org.springframework.data.redis.listener.RedisMessageListenerContainer">
<property name="connectionFactory" ref="connectionFactory"/>
<property name="messageListeners">
<map>
<entry key-ref="messageListener">
<bean class="org.springframework.data.redis.listener.ChannelTopic">
<constructor-arg value="chatroom"/>
</bean>
</entry>
</map>
</property>
</bean>Each time a message is received, the adapter will automatically and transparently perform (using the configured RedisSerializer) the conversion between the low-level format and the required object type. Any exceptions caused by method calls are caught and handled by the container (exceptions are logged by default).