数据查询顺序
一级缓存:本地缓存 -》二级缓存:redis缓存 -》数据库
本地缓存和分布式缓存
本地缓存
:基于jvm, 意思是程序放在哪,数据就存储在哪,不需要网络请求,特别快,但是需要占用jvm的内存,所以存储的东西不能太大,也不能太久,有内存淘汰机制。缓存组件包括:Guava Cache
、Caffeine
、Encache
分布式缓存
:请求需要网络传输,速度慢一点,组件包括:Redis
,MongoDB
哪些场景用到缓存?
- 邀请入会的助力排行榜,key= 活动id+pageSize,缓存过期时间1分钟
/*** 查询邀请入会的助力排行榜* @param activityId* @param pageSize* @return*/@Override@MallCache(value = "queryHelpRankingList", expireTime = 1, timeUnit = TimeUnitEnum.MINUTES)public List<ActivityHelpStatisticsEntity> queryRankingList(String activityId, Integer pageSize) {List<ActivityHelpStatisticsLogPO> statisticsLogs = mapper.queryRankingList(activityId, pageSize);if (CollectionUtils.isEmpty(statisticsLogs)) {return new ArrayList<>();}return statisticsLogs.stream().map(po -> ActivityHelpStatisticsConverter.convert(po)).collect(Collectors.toList());}
- 验证用户是否在分组中,可以=groupId, String nascentId, String groupIds,缓存五秒
@MallCache(value = "checkExistGroup", expireTime = 5, timeUnit = TimeUnitEnum.SECONDS)private boolean checkExistGroupCache(long groupId, String nascentId, String groupIds){return OpenPlatformHelperV4.checkExistGroup(groupId, nascentId, groupIds);}
- 等级体系,缓存30分钟
@MallCache(value = "GradeSystem", expireTime = 30)@Overridepublic GradeSystem queryGradeSystem(Long groupId, Long viewId, Long shopId) {GradeSystemGetRequest gradeSystemGetRequest = new GradeSystemGetRequest();gradeSystemGetRequest.setViewId(viewId);gradeSystemGetRequest.setShopId(shopId);gradeSystemGetRequest.setGroupId(groupId);GradeSystemGetResponse gradeResponse = OpenPlatformClient.exec(groupId, gradeSystemGetRequest);GradeSystem result = gradeResponse.getResult();return result;}
- 等级名称 30分钟,这种需要调用三方接口,缓存可以减少调用接口次数
/*** 获取等级名称** @param groupId* @param shopId* @param grade* @return*/@MallCache(value = "CustomerGradeName", expireTime = 30)public String getCustomerGradeName(long groupId, long shopId, int grade) {List<Long> viewIds = cloudPlatformService.queryViewIdByAreaIdOrBrandId(groupId, getMallId(), Arrays.asList(shopId));return getCustomerGradeName(groupId, shopId, grade, viewIds.get(0));}public String getCustomerGradeName(long groupId, long shopId, int grade, long viewId) {if (grade == 0) {return "非会员";}GradeInfoGetResponse response = OpenPlatformHelperV4.getGradeInfo(groupId, shopId, grade, viewId);AssertUtil.assertTrue(response != null && response.getSuccess(), "获取等级信息失败");GradeInfo gradeInfo = response.getResult();AssertUtil.assertNotNull(gradeInfo, "获取等级信息失败");return gradeInfo.getGradeName();}
@Override@MallCache(value = "VIEWID", expireTime = 60)public Long queryViewIdByAreaIdOrBrandId(Long groupId, Long mallId, Long shopId) {WmCompanyDO company = WmCompanyDao.dao().findByCompanyId(groupId);List<Long> viewIds = CloudPlatformHelper.queryViewIdByShopIds(Arrays.asList(shopId), groupId, company.getViewOperationType());AssertUtil.assertTrue(!CollectionUtils.isEmpty(viewIds), "查询运营视角失败,请检查");return viewIds.get(0);}
- 线下门店列表 缓存60分钟,这种东西一般不会经常变化,且进入首页就要用,
/**** @Description 获取线下门店列表,并存入缓存* @Param* @param groupId* @param shopId* @return*/@MallCache(value = "offlineShops", expireTime = 60)public TableResponse<Record> getOfflineShops(Long groupId,Long shopId){TableResponse<Record> tableResponse = new TableResponse<>();List<Record> offlineShops = WmOfflineShopDao.dao().getOfflineByDigitalShopId(groupId,shopId);tableResponse.setData(offlineShops);return tableResponse;}
/*** 积分别名** @param groupId* @param shopId* @param sysId* @return*/@MallCache(value = "getGameIntegralAlias", expireTime = 60)public String getIntegralAlias(Long groupId, Long mallId, Long shopId, String sysId) {Long viewId = cloudViewUtil.getViewId(groupId, mallId, shopId);return openIntegralCacheUtil.getIntegralAliasCache(groupId, mallId, shopId, sysId, viewId);}
项目中如何做缓存设计?
基于AOP的切面注解
目录结构
自定义@MallCache注解
/*** 二级缓存:redis+本地缓存* 默认启用redis缓存,本地缓存由withLocalCache控制* @author*/
@Documented
@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.METHOD)
public @interface MallCache {String value() default "MallCache::";/*** 是否启用本地缓存** @return*/boolean withLocalCache() default false;/*** 分钟级缓存:* 一级缓存的失效时间为10分钟,这里二级缓存的失效时间应该大于等于10分钟,* 不然失去二级缓存的意义* 秒级缓存:本地缓存失效时间是5s* @return 缓存失效时间 (默认分钟)*/long expireTime() default 20;/*** 缓存失效时间单位* TimeUnitEnum.MINUTES使用分钟级缓存* TimeUnitEnum.SECONDS使用秒级缓存** @return*/TimeUnitEnum timeUnit() default TimeUnitEnum.MINUTES;
}
引用注解@MallCache
/*** 查询邀请入会的助力排行榜前100* @param activityId* @param pageSize* @return*/@Override@MallCache(value = "queryHelpRankingList", expireTime = 1, timeUnit = TimeUnitEnum.MINUTES)public List<ActivityHelpStatisticsEntity> queryRankingList(String activityId, Integer pageSize) {List<ActivityHelpStatisticsLogPO> statisticsLogs = mapper.queryRankingList(activityId, pageSize);if (CollectionUtils.isEmpty(statisticsLogs)) {return new ArrayList<>();}return statisticsLogs.stream().map(po -> ActivityHelpStatisticsConverter.convert(po)).collect(Collectors.toList());}
需要得到的效果
比如 上面的queryRankingList(String activityId, Integer pageSize)
方法引用了@MallCache(value = "queryHelpRankingList", expireTime = 1, timeUnit = TimeUnitEnum.MINUTES)
注解。
最终执行过程应该是:
- 首先用
queryHelpRankingList
,activityId
,pageSize
拼接成key
,比如activityId
=hh34hrnih352nb3kh5o3g34
,pageSize
=100,那拼接的key
=queryHelpRankingList_hh34hrnih352nb3kh5o3g34_100
- 拿着key去本地查缓存,本地缓存有的话返回给前端
- 本地缓存没有就去查redis缓存,redis缓存有的话返回给前端,同时更新到本地缓存
- redis缓存没有的话,调用
queryRankingList(String activityId, Integer pageSize)
的代码去查数据库 - 同时将数据库的最新数据更新到redis和本地缓存
LocalCacheAspect
用于切面加强
@Slf4j
@Aspect
@Component
@Order(-1)
@ConditionalOnBean(CommonCacheManager.class)
public class LocalCacheAspect {@Autowiredprivate CommonCacheManager commonCacheManager;@Around("@annotation(com.nascent.ecrp.mall.core.cache.annotation.MallCache)")public Object mallCache(ProceedingJoinPoint proceedingJoinPoint){MethodSignature methodSignature = (MethodSignature) proceedingJoinPoint.getSignature();Method method = methodSignature.getMethod();Type genericReturnType = methodSignature.getMethod().getAnnotatedReturnType().getType();MallCache mallCache = method.getAnnotation(MallCache.class);String cachePrefix = mallCache.value();long expire = mallCache.expireTime();boolean withLocalCache = mallCache.withLocalCache();TimeUnitEnum timeUnit = mallCache.timeUnit();Object[] args = proceedingJoinPoint.getArgs();// 用切点方法的参数拼接成cacheKey 比如groupId_viewId_shopIdString cacheKey = Arrays.stream(args).map(arg -> {return String.valueOf(arg);}).collect(Collectors.joining("_"));Object result = commonCacheManager.queryData(cachePrefix + "_" + cacheKey,expire ,timeUnit, withLocalCache, genericReturnType ,()->{try {// 接口查询数据库拿到数据return proceedingJoinPoint.proceed(args);} catch (Throwable throwable) {log.error("LocalCacheAspect.mallCache 异常:{}", throwable);return null;}});return result;}
}
CommonCacheFactory
本地缓存的初始化类
- 初始化缓存,设置缓存的级别,最大缓存数,过期时间
- 增加缓存,删除缓存
public class CommonCacheFactory {/*** 秒级时间的缓存*/private static Cache<String, Object> localCacheSecond;/*** 分钟时间的缓存*/private static Cache<String, Object> localCacheMinute;static {localCacheSecond = Caffeine.newBuilder().maximumSize(1000).expireAfterWrite(5, TimeUnit.SECONDS).recordStats().build();localCacheMinute = Caffeine.newBuilder().maximumSize(1000).expireAfterWrite(10, TimeUnit.MINUTES).recordStats().build();}private CommonCacheFactory(){}public static Cache<String, ? > get() {return getMinute();}public static void set(String key, Object value){setMinute(key, value);}public static Cache<String, ? > getSecond() {return localCacheSecond;}public static void setSecond(String key, Object value){localCacheSecond.put(key, value);}public static Cache<String, ? > getMinute() {return localCacheMinute;}public static void setMinute(String key, Object value){localCacheMinute.put(key, value);}}
CommonCacheManager
本地缓存管理类,用于本地缓存的查询和更新
@Slf4j
@Component
public class CommonCacheManager extends AbstractCacheManager<String, String>{private static final String LOCAL_CACHE_ENABLE_KEY = "LOCAL_CACHE_ENABLE";@Autowiredprivate RedisTemplate<String, String> redisTemplate;@Overridepublic boolean useCache() {// todo 优化成直接在内存中获取String enable = redisTemplate.opsForValue().get(LOCAL_CACHE_ENABLE_KEY);if(StringUtil.isNotBlank(enable) && "CLOSE".equals(enable)){return false;}else {return true;}}@Overridepublic Cache<String, String> primaryCache(TimeUnitEnum timeUnit) {if(TimeUnitEnum.SECONDS.equals(timeUnit)){Cache<String, String> stringResultCache = (Cache<String, String>) CommonCacheFactory.getSecond();return stringResultCache;}return (Cache<String, String>) CommonCacheFactory.getMinute();}@Overridepublic String cacheKey() {return "LOCAL::CACHE::";}public <T> T queryData(String k, Long expireTime, TimeUnitEnum timeUnit, boolean withLocalCache, Type type, Supplier<T> supplier) {if (!useCache()) {// 不使用本地缓存,默认是使用的log.info("直接在缓存中获取数据:{}", k);return supplier.get();// supplier.get()能获取函数的返回值,直接返回数据库数据}try {CacheContext<String,T> resultCacheContext =CacheContext.<String,T>builder().key(cacheKey() + k).reference(type).expireSeconds(expireTime).timeUnit(timeUnit.getTimeUnit()).callback(() -> supplier.get())// 数据库数据,这里只作为实参,还并没有调用.build();// 不用本地缓存,用redis缓存if(!withLocalCache){return JSON.parseObject(getFromSecondary(resultCacheContext), type);}// 用本地缓存String result = primaryCache(timeUnit).get(k, (x) -> getFromSecondary(resultCacheContext));return JSON.parseObject(result, type);} catch (Exception e) {log.error("一级缓存获取数据失败:{}", e.getMessage());return supplier.get();}}}
AbstractCacheManager
redis缓存的管理接口类
@Slf4j
public abstract class AbstractCacheManager<K, V> implements CacheManage<K, V> {/** 查询redis缓存、存在则读取并返回,不存在则更新成最新数据** @param context* @return* @param <T>*/protected <T> V getFromSecondary(CacheContext<K,T> context) {K key = context.key;T dbValue = null;try {String valueFromSecondary = secondaryCache().getValueFromSecondary(key);// 二级缓存,即redis层缓存,有数据就使用redis缓存if (StringUtil.isNotBlank(valueFromSecondary)) {return (V) valueFromSecondary;}// redis缓存没有,查询数据库dbValue = context.getCallback().get();if (dbValue == null) {return null;}Object jsonValue = JSONObject.toJSONString(dbValue);//把数据库的数据跟新到二级缓存secondaryCache().setValueForSecondary(key, context.expireSeconds, context.timeUnit, jsonValue);return (V)jsonValue;} catch (Exception e) {log.error("二级缓存中获取数据失败:{}", e.getMessage());return null;}}@Overridepublic SecondaryCache secondaryCache() {RedisSecondaryCache redisSecondaryCache = SecondaryCacheFactory.create(RedisSecondaryCache.class);return redisSecondaryCache;}@Overridepublic void invalidate(Object o) {for(TimeUnitEnum timeUnit : TimeUnitEnum.values()){primaryCache(timeUnit).invalidate(o);}secondaryCache().invalidate(o);}@Overridepublic void invalidateAll(List o) {for(TimeUnitEnum timeUnit : TimeUnitEnum.values()){primaryCache(timeUnit).invalidateAll();}secondaryCache().invalidateAll(o);}@Overridepublic Map<String,Map<String, V>> showAsMap() {Map<String,Map<String, V>> map = Maps.newHashMap();for(TimeUnitEnum timeUnit : TimeUnitEnum.values()) {Map<String,V> newMap = Maps.newHashMap();primaryCache(timeUnit).asMap().entrySet().forEach(entry -> newMap.put(Objects.toString(entry.getKey()),entry.getValue()));map.put(timeUnit.name(), newMap);}return map;}@Overridepublic Map<String,Set<K>> keys() {Map<String,Set<K>> map = new HashMap();for(TimeUnitEnum timeUnit : TimeUnitEnum.values()) {map.put(timeUnit.name(), primaryCache(timeUnit).asMap().keySet());}return map;}}
RedisSecondaryCache
@Slf4j
@Component
public class RedisSecondaryCache implements SecondaryCache<String>{@Autowiredprivate RedisTemplate<String, Object> redisTemplate;@Overridepublic String getValueFromSecondary(String key){Object redisCache = redisTemplate.opsForValue().get(key);return redisCache == null ? null : redisCache.toString();}@Overridepublic String getValueFromSecondaryBatch(List<String> cacheKeys) {Object redisCache = redisTemplate.opsForValue().multiGet(cacheKeys);return redisCache == null ? null : redisCache.toString();}@Overridepublic <T> void setValueForSecondary(String cacheKey,long expireSeconds, TimeUnit timeUnit, T value) {redisTemplate.opsForValue().set(cacheKey,JSONObject.toJSON(value),expireSeconds, timeUnit);}@Overridepublic <T> void setValueForSecondaryBatch(Map<String, T> kv, long expireSeconds) {kv.forEach((k, v) -> {redisTemplate.opsForValue().set(k, v, expireSeconds, TimeUnit.MINUTES);});}@Overridepublic void invalidate(Object o) {redisTemplate.delete(String.valueOf(o));}@Overridepublic void invalidateAll(List keys) {redisTemplate.delete(keys);}}
CacheContext
@Data
@NoArgsConstructor
@AllArgsConstructor
@Builder
public class CacheContext<K, T> {K key;Type reference;// 传递函数,Supplier可以把函数作为实参传递,调用Supplier.get()时返回函数的结果Supplier<T> callback;Long expireSeconds;TimeUnit timeUnit;
}
CacheManage
缓存管理接口
-定义
public interface CacheManage<K, V> extends InvalidateCommand {boolean useCache();String cacheKey();Cache<K, V> primaryCache(TimeUnitEnum timeUnit);SecondaryCache secondaryCache();Map<String, Map<String, V>> showAsMap();Map<String,Set<K>> keys();}
SecondaryCache
redis缓存接口类
- 定义一些接口,比如获取缓存,设置缓存
public interface SecondaryCache<K> extends InvalidateCommand{String getValueFromSecondary(K key);String getValueFromSecondaryBatch(List<K> keys);<T> void setValueForSecondary(K key, long expireSeconds, TimeUnit timeUnit, T value);<T> void setValueForSecondaryBatch( Map<K, T > kv, long expireSeconds);
}
SecondaryCacheFactory
二级缓存工厂类
- 感觉啥也没干,不知道这个有什么用
public class SecondaryCacheFactory {private SecondaryCacheFactory(){}public static <T> T create(Class<T> clazz) {return (T) SpringContext.me().getBean(clazz);}
}
TimeUnitEnum
/*** 缓存支持的时间的单位* @author*/
public enum TimeUnitEnum {MINUTES(TimeUnit.MINUTES),SECONDS(TimeUnit.SECONDS);@Getterprivate TimeUnit timeUnit;TimeUnitEnum(TimeUnit timeUnit){this.timeUnit = timeUnit;}
}
InvalidateCommand
public interface InvalidateCommand<K> {void invalidate(K k);void invalidateAll(List<K> k);
}