Spring cache CompositeCacheManager example

Spring cache多缓存实现整合

前言

我参与的微服务项目一直使用Redis作为分布式缓存,代码上使用Spring cache和Jedis连接Redis。
前一段时间发现有一些数据使用的很频繁却又极少被改变,而将数据放到Redis上,就出现了浪费网络带宽的问题。
这个数据的大小大概有300Kb,而且很少有内容变动,在秒并发500的情况下,每一个请求都要去Redis获取这数据,网络流量为150Mb/s,在千兆内网网卡中就占了15%。
究其原因,是因为该数据量较大(达到了300Kb),同时该数据使用频繁(每个请求都需要获取),然后内容很少变动。

解决方式

期望在项目中引入内存缓存,首选是EhCache。
这样每个请求在连接redis前,先到ehcache中看看是否有缓存,没有的话再去redis上拿取缓存。redis上也没有才去数据库中select。

遇到的问题

项目使用Spring 4.0.5,而这个版本的spring cache的@Cacheable等注解并不支持选择缓存类型。这个特性需要Spring 4.1.x,项目又不能轻易升级spring版本。

解决方案

使用CompositeCacheManager来解决问题. 可以看到如下样例中, 重写的CachingConfigurer的cacheManager()方法提供了一个CompositeCacheManager类. 类中指定了ehCacheManager()和redisCacheManager()两个manager.
其中ehCacheManager读取了/cache/ehcache.xml这个配置文件初始化了ehcache, 而redisCacheManager使用Jedis连接了redis.
这样就可以实现同时使用ehcache和redis了, 并且都可以使用Spring提供的@Cacheable等注解.
在new CompositeCacheManager类时指定的顺序是先ehcache后redis, 因此@Cacheable等注解指定的key在ehcache的xml配置文件中不存在的话, 就会使用redis.
比如样例中, 一共有四个key, EHCACHE_KEY和EHCACHE_IDENTIFY_KEY在ehcache.xml中, 而REDIS_KEY和REDIS_IDENTIFY_KEY没有. 因此划分出两个key将会存放在ehcache, 两个key将会放到redis.

样例

CompositeCacheManagerTest

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
@ActiveProfiles("test")
@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(classes = {TestConfig.class})
@WebAppConfiguration
public class CompositeCacheManagerTest {

final Logger logger = LoggerFactory.getLogger(getClass());
public static final FastDateFormat DATETIME_FORMAT = FastDateFormat.getInstance("yyyy-MM-dd'T'HH:mm:ss:SSS");
public static final String REDIS_KEY = "redis-key";
public static final String EHCACHE_KEY = "ehcache-key";
public static final String REDIS_IDENTIFY_KEY = "redis-identify-key";
public static final String EHCACHE_IDENTIFY_KEY = "ehcache-identify-key";
public static final int SLEEP_MILLIS = 300;
@Autowired
private WebApplicationContext webApplicationContext;

protected MockMvc mockMvc;

@Before
public void createMockMVC() {
mockMvc = MockMvcBuilders.webAppContextSetup(webApplicationContext).addFilter(new GlobalFilter())
.build();
}

@Test
public void applicationContextShouldBeInitialized() {
Assert.assertNotNull(webApplicationContext);
for (String beanDefinitionNames : webApplicationContext.getBeanDefinitionNames()) {
Assert.assertNotNull(webApplicationContext.getBean(beanDefinitionNames));
}
}

@Autowired
StringRedisTemplate stringRedisTemplate;
@Autowired
RedisCacheService redisCacheService;
@Autowired
EhCacheService ehCacheService;
@Autowired
CacheManager ehCacheManager;
@Autowired
CacheManager redisCacheManager;

@Test
public void testEhCacheLoad() throws Exception {
Collection<String> cacheNames = ehCacheManager.getCacheNames();
logger.debug("cacheNames={}", cacheNames);
Assert.assertThat(cacheNames, Matchers.contains(EHCACHE_KEY, EHCACHE_IDENTIFY_KEY));
}

@Test
public void testRedisCacheLoad() throws Exception {
Collection<String> cacheNames = redisCacheManager.getCacheNames();
logger.debug("cacheNames={}", cacheNames);
Assert.assertThat(cacheNames, Matchers.contains(REDIS_KEY, REDIS_IDENTIFY_KEY));
}

@Before
public void setUp() throws Exception {
redisCacheService.evictAll();
ehCacheService.evictAll();
}

@After
public void tearDown() throws Exception {
redisCacheService.evictAll();
ehCacheService.evictAll();
}

@Test
public void testRedis() throws Exception {
logger.debug("start test redis");
redisCacheService.clearExecuteTime();
String currentDateStringBefore = redisCacheService.get();
logger.debug("currentDateStringBefore={}", currentDateStringBefore);
Thread.currentThread().join(SLEEP_MILLIS);
String currentDateStringAfter = redisCacheService.get();
logger.debug("currentDateStringAfter={}", currentDateStringAfter);
Assert.assertEquals(currentDateStringAfter, currentDateStringBefore);
int executeTime = redisCacheService.getExecuteTime();
logger.debug("executeTime={}", executeTime);
Assert.assertEquals(executeTime, 1);
logger.debug("finish test redis");
}

@Test
public void testEhcache() throws Exception {
logger.debug("start test ehcache");
ehCacheService.clearExecuteTime();
String currentDateStringBefore = ehCacheService.get();
logger.debug("currentDateStringBefore={}", currentDateStringBefore);
Thread.currentThread().join(SLEEP_MILLIS);
String currentDateStringAfter = ehCacheService.get();
logger.debug("currentDateStringAfter={}", currentDateStringAfter);
Assert.assertEquals(currentDateStringAfter, currentDateStringBefore);
int executeTime = ehCacheService.getExecuteTime();
logger.debug("executeTime={}", executeTime);
Assert.assertEquals(executeTime, 1);
logger.debug("finish test ehcache");
}

@Test
public void testRedisIdentify() throws Exception {
logger.debug("start test redis identify");
redisCacheService.clearExecuteTime();
Long identify1 = 1L;
Long identify2 = 2L;

String value1Before = redisCacheService.get(identify1);
logger.debug("value1Before={}", value1Before);

String value2Before = redisCacheService.get(identify2);
logger.debug("value2Before={}", value2Before);

Thread.currentThread().join(SLEEP_MILLIS);

String value1After = redisCacheService.get(identify1);
logger.debug("value1After={}", value1After);
Assert.assertEquals(value1Before, value1After);

String value2After = redisCacheService.get(identify2);
logger.debug("value2After={}", value2After);
Assert.assertEquals(value2Before, value2After);

int executeTime = redisCacheService.getExecuteTime();
logger.debug("executeTime={}", executeTime);
Assert.assertEquals(executeTime, 2);

logger.debug("finish test redis identify");
}

@Test
public void testEhcacheIdentify() throws Exception {
logger.debug("start test ehcache identify");
ehCacheService.clearExecuteTime();
Long identify1 = 1L;
Long identify2 = 2L;

String value1Before = ehCacheService.get(identify1);
logger.debug("value1Before={}", value1Before);

String value2Before = ehCacheService.get(identify2);
logger.debug("value2Before={}", value2Before);

Thread.currentThread().join(SLEEP_MILLIS);

String value1After = ehCacheService.get(identify1);
logger.debug("value1After={}", value1After);
Assert.assertEquals(value1Before, value1After);

String value2After = ehCacheService.get(identify2);
logger.debug("value2After={}", value2After);
Assert.assertEquals(value2Before, value2After);

int executeTime = ehCacheService.getExecuteTime();
logger.debug("executeTime={}", executeTime);
Assert.assertEquals(executeTime, 2);

logger.debug("finish test ehcache identify");
}
}

Configuration

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
@Configuration
@ComponentScan(basePackageClasses = TestConfig.class)
@EnableAspectJAutoProxy
@EnableCaching
public class TestConfig implements CachingConfigurer {

@Bean
@Override
public CacheManager cacheManager() {
CompositeCacheManager compositeCacheManager = new CompositeCacheManager(ehCacheManager(), redisCacheManager());
compositeCacheManager.setFallbackToNoOpCache(true);
compositeCacheManager.afterPropertiesSet();
return compositeCacheManager;
}

@Bean
@Override
public KeyGenerator keyGenerator() {
return new SimpleKeyGenerator();
}

// need support in spring 4.1
//@Bean
//@Override
//public CacheResolver cacheResolver() {
// return new SimpleCacheResolver(cacheManager());
//}

// need support in spring 4.1
//@Bean
//@Override
//public CacheErrorHandler errorHandler() {
// return new SimpleCacheErrorHandler();
//}

@Bean
public CacheManager redisCacheManager() {
RedisCacheManager redisCacheManager = new RedisCacheManager(redisTemplate());
redisCacheManager.setUsePrefix(true);
redisCacheManager.setDefaultExpiration(3600);
return redisCacheManager;
}

@Bean
public JedisPoolConfig jedisPoolConfig() {
JedisPoolConfig jedisPoolConfig = new JedisPoolConfig();
jedisPoolConfig.setMaxIdle(32);
return jedisPoolConfig;
}

@Bean
public JedisConnectionFactory connectionFactory() {
JedisConnectionFactory jedisConnectionFactory = new JedisConnectionFactory();
jedisConnectionFactory.setHostName("192.168.70.209");
jedisConnectionFactory.setPort(16379);
jedisConnectionFactory.setTimeout(5000);
jedisConnectionFactory.setPoolConfig(jedisPoolConfig());
jedisConnectionFactory.setUsePool(true);
jedisConnectionFactory.setDatabase(3);
jedisConnectionFactory.afterPropertiesSet();
return jedisConnectionFactory;
}

@Bean
public StringRedisTemplate stringRedisTemplate() {
StringRedisTemplate stringRedisTemplate = new StringRedisTemplate();
stringRedisTemplate.setConnectionFactory(connectionFactory());
stringRedisTemplate.afterPropertiesSet();
return stringRedisTemplate;
}

@Bean
public RedisTemplate<Object, Object> redisTemplate() {
RedisTemplate<Object, Object> redisTemplate = new RedisTemplate<>();
redisTemplate.setConnectionFactory(connectionFactory());
Jackson2JsonRedisSerializer<Object> serializer = new Jackson2JsonRedisSerializer<>(Object.class);
serializer.setObjectMapper(ObjectMapperCreator.createDefaultObjectMapper());
redisTemplate.setKeySerializer(serializer);
redisTemplate.setValueSerializer(serializer);
redisTemplate.setHashKeySerializer(serializer);
redisTemplate.setHashValueSerializer(serializer);
return redisTemplate;
}

@Bean
public CacheManager ehCacheManager() {
return new EhCacheCacheManager(ehCacheManagerFactoryBean().getObject());
}

@Bean
public EhCacheManagerFactoryBean ehCacheManagerFactoryBean() {
EhCacheManagerFactoryBean factoryBean = new EhCacheManagerFactoryBean();
factoryBean.setConfigLocation(new ClassPathResource("/cache/ehcache.xml"));
factoryBean.setShared(true);
return factoryBean;
}

}

ehcache.xml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
<?xml version="1.0" encoding="UTF-8"?>
<ehcache updateCheck="false" monitoring="autodetect" dynamicConfig="true">
<!--
name:缓存名称。
maxElementsInMemory:缓存最大个数。
eternal:对象是否永久有效,一但设置了,timeout将不起作用。
timeToIdleSeconds:设置对象在失效前的允许闲置时间(单位:秒)。
仅当eternal=false对象不是永久有效时使用,可选属性,默认值是0,也就是可闲置时间无穷大。
timeToLiveSeconds:设置对象在失效前允许存活时间(单位:秒)。最大时间介于创建时间和失效时间之间。
仅当eternal=false对象不是永久有效时使用,默认是0.,也就是对象存活时间无穷大。
overflowToDisk:当内存中对象数量达到maxElementsInMemory时,Ehcache将会对象写到磁盘中。
diskSpoolBufferSizeMB:这个参数设置DiskStore(磁盘缓存)的缓存区大小。默认是30MB。每个Cache都应该有自己的一个缓冲区。
maxElementsOnDisk:硬盘最大缓存个数。
diskPersistent:是否缓存虚拟机重启期数据 Whether the disk store persists between restarts
of the Virtual Machine. The default value is false.
diskExpiryThreadIntervalSeconds:磁盘失效线程运行时间间隔,默认是120秒。
memoryStoreEvictionPolicy:当达到maxElementsInMemory限制时,Ehcache将会根据指定的策略去清理内存。
默认策略是LRU(最近最少使用)。你可以设置为FIFO(先进先出)或是LFU(较少使用)。
clearOnFlush:内存数量最大时是否清除。
-->
<diskStore path="java.io.tmpdir"/>

<defaultCache
maxElementsInMemory="100"
eternal="false"
timeToIdleSeconds="3600"
timeToLiveSeconds="3600"
overflowToDisk="true"
maxElementsOnDisk="10000000"
diskPersistent="false"
diskExpiryThreadIntervalSeconds="120"
memoryStoreEvictionPolicy="LRU"/>
<cache name="ehcache-key"
maxElementsInMemory="10000"
eternal="false"
timeToIdleSeconds="3600"
timeToLiveSeconds="3600"
overflowToDisk="false"
diskPersistent="false"
diskExpiryThreadIntervalSeconds="1"/>
<cache name="ehcache-identify-key"
maxElementsInMemory="10000"
eternal="false"
timeToIdleSeconds="3600"
timeToLiveSeconds="3600"
overflowToDisk="false"
diskPersistent="false"
diskExpiryThreadIntervalSeconds="1"/>
</ehcache>

EhCacheService

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
@Service
public class EhCacheService {

private int executeTime = 0;

public void clearExecuteTime() {
executeTime = 0;
}

public int getExecuteTime() {
return executeTime;
}

@Cacheable(CompositeCacheManagerTest.EHCACHE_KEY)
public String get() {
executeTime++;
return CompositeCacheManagerTest.DATETIME_FORMAT.format(new Date());
}

@Cacheable(value = CompositeCacheManagerTest.EHCACHE_IDENTIFY_KEY, key = "#identify")
public String get(Long identify) {
executeTime++;
return CompositeCacheManagerTest.DATETIME_FORMAT.format(new Date()) + "&identify:" + identify;
}

@Caching(evict = {
@CacheEvict(CompositeCacheManagerTest.EHCACHE_KEY),
@CacheEvict(value = CompositeCacheManagerTest.EHCACHE_IDENTIFY_KEY, allEntries = true)})
public void evictAll() {
}
}

RedisCacheService

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
@Service
public class RedisCacheService {

private int executeTime = 0;

public void clearExecuteTime() {
executeTime = 0;
}

public int getExecuteTime() {
return executeTime;
}

@Cacheable(CompositeCacheManagerTest.REDIS_KEY)
public String get() {
executeTime++;
return CompositeCacheManagerTest.DATETIME_FORMAT.format(new Date());
}

@Cacheable(value = CompositeCacheManagerTest.REDIS_IDENTIFY_KEY, key = "#identify")
public String get(Long identify) {
executeTime++;
return CompositeCacheManagerTest.DATETIME_FORMAT.format(new Date()) + "&identify:" + identify;
}

@Caching(evict = {
@CacheEvict(CompositeCacheManagerTest.REDIS_KEY),
@CacheEvict(value = CompositeCacheManagerTest.REDIS_IDENTIFY_KEY, allEntries = true)})
public void evictAll() {
}
}

单测运行结果

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
14:33:53.127 [main] DEBUG c.n.c.f.c.m.CompositeCacheManagerTest - start test ehcache
14:33:53.129 [main] DEBUG c.n.c.f.c.m.CompositeCacheManagerTest - currentDateStringBefore=2016-03-29T14:33:53:128
14:33:53.429 [main] DEBUG c.n.c.f.c.m.CompositeCacheManagerTest - currentDateStringAfter=2016-03-29T14:33:53:128
14:33:53.429 [main] DEBUG c.n.c.f.c.m.CompositeCacheManagerTest - executeTime=1
14:33:53.429 [main] DEBUG c.n.c.f.c.m.CompositeCacheManagerTest - finish test ehcache

14:33:53.451 [main] DEBUG c.n.c.f.c.m.CompositeCacheManagerTest - start test redis
14:33:53.463 [main] DEBUG c.n.c.f.c.m.CompositeCacheManagerTest - currentDateStringBefore=2016-03-29T14:33:53:453
14:33:53.776 [main] DEBUG c.n.c.f.c.m.CompositeCacheManagerTest - currentDateStringAfter=2016-03-29T14:33:53:453
14:33:53.776 [main] DEBUG c.n.c.f.c.m.CompositeCacheManagerTest - executeTime=1
14:33:53.776 [main] DEBUG c.n.c.f.c.m.CompositeCacheManagerTest - finish test redis

14:33:53.801 [main] DEBUG c.n.c.f.c.m.CompositeCacheManagerTest - cacheNames=[ehcache-key, ehcache-identify-key]
14:33:53.833 [main] DEBUG c.n.c.f.c.m.CompositeCacheManagerTest - cacheNames=[redis-key, redis-identify-key]

14:33:53.858 [main] DEBUG c.n.c.f.c.m.CompositeCacheManagerTest - start test redis identify
14:33:53.892 [main] DEBUG c.n.c.f.c.m.CompositeCacheManagerTest - value1Before=2016-03-29T14:33:53:890&identify:1
14:33:53.895 [main] DEBUG c.n.c.f.c.m.CompositeCacheManagerTest - value2Before=2016-03-29T14:33:53:894&identify:2
14:33:54.197 [main] DEBUG c.n.c.f.c.m.CompositeCacheManagerTest - value1After=2016-03-29T14:33:53:890&identify:1
14:33:54.199 [main] DEBUG c.n.c.f.c.m.CompositeCacheManagerTest - value2After=2016-03-29T14:33:53:894&identify:2
14:33:54.199 [main] DEBUG c.n.c.f.c.m.CompositeCacheManagerTest - executeTime=2
14:33:54.199 [main] DEBUG c.n.c.f.c.m.CompositeCacheManagerTest - finish test redis identify

14:33:54.231 [main] DEBUG c.n.c.f.c.m.CompositeCacheManagerTest - start test ehcache identify
14:33:54.232 [main] DEBUG c.n.c.f.c.m.CompositeCacheManagerTest - value1Before=2016-03-29T14:33:54:231&identify:1
14:33:54.232 [main] DEBUG c.n.c.f.c.m.CompositeCacheManagerTest - value2Before=2016-03-29T14:33:54:232&identify:2
14:33:54.532 [main] DEBUG c.n.c.f.c.m.CompositeCacheManagerTest - value1After=2016-03-29T14:33:54:231&identify:1
14:33:54.532 [main] DEBUG c.n.c.f.c.m.CompositeCacheManagerTest - value2After=2016-03-29T14:33:54:232&identify:2
14:33:54.532 [main] DEBUG c.n.c.f.c.m.CompositeCacheManagerTest - executeTime=2
14:33:54.532 [main] DEBUG c.n.c.f.c.m.CompositeCacheManagerTest - finish test ehcache identify
文章目录
  1. 1. Spring cache多缓存实现整合
    1. 1.1. 前言
    2. 1.2. 解决方式
    3. 1.3. 遇到的问题
    4. 1.4. 解决方案
    5. 1.5. 样例
      1. 1.5.1. CompositeCacheManagerTest
      2. 1.5.2. Configuration
      3. 1.5.3. ehcache.xml
      4. 1.5.4. EhCacheService
      5. 1.5.5. RedisCacheService
      6. 1.5.6. 单测运行结果
|