基于javaPoet的缓存key优化实践

360影视 2025-01-14 13:31 2

摘要:在一次系统opsreview中,发现了一些服务配置了@Cacheable注解。@cacheable 来源于Spring cache框架中,作用是使用aop的方式将数据库中的热数据缓存在redis/本地缓存中,代码如下:

在一次系统opsreview中,发现了一些服务配置了@Cacheable注解。@cacheable 来源于Spring cache框架中,作用是使用aop的方式将数据库中的热数据缓存在redis/本地缓存中,代码如下:

@Cacheable(value = { "per" }, key="#person.getId"+"_"+"#person.getName")public Person getByIsbn(Person person) { return personMapper.getPerson(person);}

那么这个原生spring组件是如何工作的?redis的key是如何产生的?这一过程是否还有优化的空间?带着这些问题我们来开启源码之旅。

就以项目中使用的spring3.2.18版本为例分析,代码中使用了xml+cache标签的形式去启动注解缓存。然而在springboot中使用的是@EnableCaching注解,通过自动配置加载相关组件,两种方式都是殊途同归,这里就不做赘述了,直接上链接。

首先,如果我们想使用这个组件就需要先启用缓存注解,方式与aop功能相类似,aop也会加载internalAutoProxyCreator后置处理器。代码中通过annotation-driven标签加载相关组件。其中proxy-target-class="true" 表示使用CGLIB的方式对bean进行动态代理。

//

代码中cache-manager表示需要依赖一个缓存管理器,它的作用是提供一种机制来缓存数据,以便在后续的访问中可以更快地获取数据。它可以支持caffine,encache,Jcache等多种类型的缓存管理器。文中是使用的自定义管理来支持公司内部的redis客户端。



//redis缓存管理器public class RedisCacheManager extends AbstracttransactionSupportingCacheManager { private Collection caches; public void setCaches(List caches) { this.Caches = caches; } @Override protected Collection loadCaches { if (caches == null) { return Collections.emptyList; } return caches; } @Override public Cache getCache(String name) { Cache cache = super.getCache(name); if (cache == null && (cache = super.getCache("DEFAULT")) == null) { throw new NullPointerException; } return cache; }}

下面通过bean的方式注入cacheManager管理器,其中MyCache需要实现org.springframework.cache.Cache中定义的方法,以达到手动diy缓存操作的目的。

Cache接口中有get,put,evict等方法,可以按需替换成自己想要的操作。

public interface Cache { String getName; Object getNativeCache; Cache.ValueWrapper get(Object var1); void put(Object var1, Object var2); void evict(Object var1); void clear; public interface ValueWrapper { Object get; }}

配置输出完了,开始切入正题。spring容器启动时候会解析annotation-driven标签,具体的实现在CacheNamespaceHandler中。显然可以发现beanDefinition解析类是AnnotationDrivenCacheBeanDefinitionParser。

public class CacheNamespaceHandler extends NamespaceHandlerSupport { static final String CACHE_MANAGER_ATTRIBUTE = "cache-manager"; static final String DEFAULT_CACHE_MANAGER_BEAN_NAME = "cacheManager"; public CacheNamespaceHandler { } static String extractCacheManager(Element element) { return element.hasAttribute("cache-manager") ? element.getAttribute("cache-manager") : "cacheManager"; } static BeanDefinition parseKeyGenerator(Element element, BeanDefinition def) { String name = element.getAttribute("key-generator"); if (StringUtils.hasText(name)) { Def.getPropertyValues.add("keyGenerator", new RuntimeBeanReference(name.trim)); } return def; } public void init { this.registerBeanDefinitionParser("annotation-driven", new AnnotationDrivenCacheBeanDefinitionParser); this.registerBeanDefinitionParser("advice", new CacheAdviceParser); }}

AnnotationDrivenCacheBeanDefinitionParser中会先判断生成切面的方式,默认使用spring原生aop,也可以通过mode标签切换成AspectJ。

public BeanDefinition parse(Element element, ParserContext parserContext) { String mode = element.getAttribute("mode"); if ("aspectj".equals(mode)) { this.registerCacheAspect(element, parserContext); } else { AnnotationDrivenCacheBeanDefinitionParser.AopAutoProxyConfigurer.configureAutoProxyCreator(element, parserContext); } return null; }

往下走会到达configureAutoProxyCreator方法,configureAutoProxyCreator方法的作用是配置自动代理创建器。代码很多继续往下看~

public static void configureAutoProxyCreator(Element element, ParserContext parserContext) { AopNamespaceUtils.registerAutoProxyCreatorifNecessary(parserContext, element); if (!parserContext.getRegistry.containsBeanDefinition("org.springframework.cache.config.internalCacheAdvisor")) { Object eleSource = parserContext.extractSource(element); RootBeanDefinition sourceDef = new RootBeanDefinition("org.springframework.cache.annotation.AnnotationCacheOperationSource"); sourceDef.setSource(eleSource); sourceDef.setRole(2); String sourceName = parserContext.getReaderContext.registerWithGeneratedName(sourceDef); RootBeanDefinition interceptorDef = new RootBeanDefinition(CacheInterceptor.class); interceptorDef.setSource(eleSource); interceptorDef.setRole(2); AnnotationDrivenCacheBeanDefinitionParser.parseCacheManagerProperty(element, interceptorDef); CacheNamespaceHandler.parseKeyGenerator(element, interceptorDef); interceptorDef.getPropertyValues.add("cacheOperationSources", new RuntimeBeanReference(sourceName)); String interceptorName = parserContext.getReaderContext.registerWithGeneratedName(interceptorDef); RootBeanDefinition advisorDef = new RootBeanDefinition(BeanFactoryCacheOperationSourceAdvisor.class); advisorDef.setSource(eleSource); advisorDef.setRole(2); advisorDef.getPropertyValues.add("cacheOperationSource", new RuntimeBeanReference(sourceName)); advisorDef.getPropertyValues.add("adviceBeanName", interceptorName); if (element.hasAttribute("order")) { advisorDef.getPropertyValues.add("order", element.getAttribute("order")); } parserContext.getRegistry.registerBeanDefinition("org.springframework.cache.config.internalCacheAdvisor", advisorDef); CompositeComponentDefinition compositeDef = new CompositeComponentDefinition(element.getTagName, eleSource); compositeDef.addNestedComponent(new BeanComponentDefinition(sourceDef, sourceName)); compositeDef.addNestedComponent(new BeanComponentDefinition(interceptorDef, interceptorName)); compositeDef.addNestedComponent(new BeanComponentDefinition(advisorDef, "org.springframework.cache.config.internalCacheAdvisor")); parserContext.registerComponent(compositeDef); } }

AopNamespaceUtils.registerAutoProxyCreatorIfNecessary(parserContext, element)作用是注册动态代理创建器。跳转两次到达这个registerOrEscalateApcAsRequired方法,它会检查是否存在org.springframework.aop.config.internalAutoProxyCreator的beanDefinition。

大概意思就是检查此前是否还有其他的代理比如aop代理,它也会加载internalAutoProxyCreator这个后置处理器。如果已经加载过internalAutoProxyCreator,则根据自动代理创建器的优先级判断,使用优先级高者。然后返回internalAutoProxyCreator的beanDefinition。

private static BeanDefinition registerOrEscalateApcAsRequired(Class cls, BeanDefinitionRegistry registry, Object source) { Assert.notNull(registry, "BeanDefinitionRegistry must not be null"); if (registry.containsBeanDefinition("org.springframework.aop.config.internalAutoProxyCreator")) { BeanDefinition apcDefinition = registry.getBeanDefinition("org.springframework.aop.config.internalAutoProxyCreator"); if (!cls.getName.equals(apcDefinition.getBeanClassName)) { int currentPriority = findPriorityForClass(apcDefinition.getBeanClassName); int requiredPriority = findPriorityForClass(cls); if (currentPriority

书接上文,获取beanDefinition后,会根据配置查看bean代理生成使用哪种模式,上文提到了,这里会根据proxy-target-class属性做判断,如果为true则使用CGLIB。添加属性配置后会调用registerComponentIfNecessary重新注册internalAutoProxyCreator组件。

private static void useClassProxyingIfNecessary(BeanDefinitionRegistry registry, Element sourceElement) { if (sourceElement != null) { boolean proxyTargetClass = Boolean.valueOf(sourceElement.getAttribute("proxy-target-class")); if (proxyTargetClass) { AopConfigUtils.forceAutoProxyCreatorToUseClassProxying(registry); } boolean exposeProxy = Boolean.valueOf(sourceElement.getAttribute("expose-proxy")); if (exposeProxy) { AopConfigUtils.forceAutoProxyCreatorToExposeProxy(registry); } } } private static void registerComponentIfNecessary(BeanDefinition beanDefinition, ParserContext parserContext) { if (beanDefinition != null) { BeanComponentDefinition componentDefinition = new BeanComponentDefinition(beanDefinition, "org.springframework.aop.config.internalAutoProxyCreator"); parserContext.registerComponent(componentDefinition); } }

回到主流程中首先判断是否加载过org.springframework.cache.config.internalCacheAdvisor目的是避免重复。校验过后定义了AnnotationCacheOperationSource这个beanDefinition,这个类比较绕,通过上帝视角总结下,它的作用是解析目标方法中包含了哪些缓存操作, 比如Cacheable等注解。后面会作为其他bean的成员变量。

RootBeanDefinition sourceDef = new RootBeanDefinition("org.springframework.cache.annotation.AnnotationCacheOperationSource"); sourceDef.setSource(eleSource); sourceDef.setRole(2); String sourceName = parserContext.getReaderContext.registerWithGeneratedName(sourceDef);

接下来,是CacheInterceptor类的beanDefinition注册。CacheInterceptor实现了aop的MethodInterceptor接口,我们可以叫他代理中的代理。。。

创建beanDefinition后将前文中AnnotationCacheOperationSource解析器作为配置项添加到CacheInterceptor的bean定义中。

RootBeanDefinition interceptorDef = new RootBeanDefinition(CacheInterceptor.class); interceptorDef.setSource(eleSource); interceptorDef.setRole(2); //这块不特别说明了,目的是为了添加cacheManager ref AnnotationDrivenCacheBeanDefinitionParser.parseCacheManagerProperty(element, interceptorDef); //设置KeyGenerator,不够灵活pass掉了 CacheNamespaceHandler.parseKeyGenerator(element, interceptorDef); // interceptorDef.getPropertyValues.add("cacheOperationSources", new RuntimeBeanReference(sourceName));

CacheInterceptor实际的作用是为配置@Cacheable注解的目标方法提供切面功能,非常类似于一个定制化的@around。直接上代码。通过上面的解析器获取出缓存操作列表,如果能获取到缓存且不需要更新缓存则直接返回数据。如果需要更新则通过目标方法获取最新数据,在刷新缓存后直接返回。在这里包含了生成rediskey的步骤,后面会有介绍。

protected Object execute(CacheAspectSupport.Invoker invoker, Object target, Method method, Object args) { if (!this.initialized) { return invoker.invoke; } else { Class targetClass = AopProxyUtils.ultimateTargetClass(target); if (targetClass == null && target != null) { targetClass = target.getClass; } Collection cacheOp = this.getCacheOperationSource.getCacheOperations(method, targetClass); if (!CollectionUtils.isEmpty(cacheOp)) { Map> ops = this.createOperationContext(cacheOp, method, args, target, targetClass); this.inspectBeforeCacheEvicts((Collection)ops.get("cacheevict")); CacheAspectSupport.CacheStatus status = this.inspectCacheables((Collection)ops.get("cacheable")); Map updates = this.inspectCacheUpdates((Collection)ops.get("cacheupdate")); if (status != null) { if (!status.updateRequired) { return status.retVal; } updates.putAll(status.cacheUpdates); } Object retVal = invoker.invoke; this.inspectAfterCacheEvicts((Collection)ops.get("cacheevict"), retVal); if (!updates.isEmpty) { this.update(updates, retVal); } return retVal; } else { return invoker.invoke; } } }

返回主流程,下面这部分是BeanFactoryCacheOperationSourceAdvisor缓存通知器的beanDefinition。这个类功能是注册aop,声明了切面的连接点(实际上依赖于上文中cacheOperationSource这个bean)与通知(实际上依赖于上文中CacheInterceptor这个bean)。

RootBeanDefinition advisorDef = new RootBeanDefinition(BeanFactoryCacheOperationSourceAdvisor.class); advisorDef.setSource(eleSource); advisorDef.setRole(2); advisorDef.getPropertyValues.add("cacheOperationSource", new RuntimeBeanReference(sourceName)); advisorDef.getPropertyValues.add("adviceBeanName", interceptorName); if (element.hasAttribute("order")) { advisorDef.getPropertyValues.add("order", element.getAttribute("order")); } parserContext.getRegistry.registerBeanDefinition("org.springframework.cache.config.internalCacheAdvisor", advisorDef);

BeanFactoryCacheOperationSourceAdvisor类实现了PointcutAdvisor指定了切面点(实际没用表达式,直接通过match暴力获取注解,能获取到则表示命中aop)

public class BeanFactoryCacheOperationSourceAdvisor extends AbstractBeanFactoryPointcutAdvisor { private CacheOperationSource cacheOperationSource; private final CacheOperationSourcePointcut pointcut = new CacheOperationSourcePointcut { protected CacheOperationSource getCacheOperationSource { return BeanFactoryCacheOperationSourceAdvisor.this.cacheOperationSource; } }; public BeanFactoryCacheOperationSourceAdvisor { } public void setCacheOperationSource(CacheOperationSource cacheOperationSource) { this.cacheOperationSource = cacheOperationSource; } public void setClassFilter(ClassFilter classFilter) { this.pointcut.setClassFilter(classFilter); } public Pointcut getPointcut { return this.pointcut; }}//其中切面点matchs方法public boolean matches(Method method, Class targetClass) { CacheOperationSource cas = this.getCacheOperationSource; return cas != null && !CollectionUtils.isEmpty(cas.getCacheOperations(method, targetClass)); }

最后,注册复合组件,并将其注册到解析器上下文中。熟悉aop源码就可以知道,在bean实例化阶段,后置处理器会检查bean命中了哪个aop,再根据自动代理生成器中的配置,来决定使用哪种代理方式生成代理类,同时织入对应的advice。实际上是代理到CacheInterceptor上面,CacheInterceptor中间商内部再调用target目标类,就是这么简单~

CompositeComponentDefinition compositeDef = new CompositeComponentDefinition(element.getTagName, eleSource); compositeDef.addNestedComponent(new BeanComponentDefinition(sourceDef, sourceName)); compositeDef.addNestedComponent(new BeanComponentDefinition(interceptorDef, interceptorName)); compositeDef.addNestedComponent(new BeanComponentDefinition(advisorDef, "org.springframework.cache.config.internalCacheAdvisor")); parserContext.registerComponent(compositeDef);

然而key是如何产生的?通过上问的阐述,就知道要找这个中间商CacheInterceptor,上代码。

倒车回到这里,最直观的嫌疑人是return status.retVal;这句继续跟进status。

private CacheAspectSupport.CacheStatus inspectCacheables(Collection cacheables) { Map cacheUpdates = new LinkedHashMap(cacheables.size); boolean cacheHit = false; Object retVal = null; if (!cacheables.isEmpty) { boolean log = this.logger.isTraceEnabled; boolean atLeastOnePassed = false; Iterator i$ = cacheables.iterator; while(true) { while(true) { CacheAspectSupport.CacheOperationContext context; Object key; label48: do { while(i$.hasNext) { context = (CacheAspectSupport.CacheOperationContext)i$.next; if (context.isConditionPassing) { atLeastOnePassed = true; key = context.generateKey; if (log) { this.logger.trace("Computed cache key " + key + " for operation " + context.operation); } if (key == null) { throw new IllegalArgumentException("Null key returned for cache operation (maybe you are using named params on classes without debug info?) " + context.operation); } cacheUpdates.put(context, key); continue label48; } if (log) { this.logger.trace("Cache condition failed on method " + context.method + " for operation " + context.operation); } } if (atLeastOnePassed) { return new CacheAspectSupport.CacheStatus(cacheUpdates, !cacheHit, retVal); } return null; } while(cacheHit); Iterator i$ = context.getCaches.iterator; while(i$.hasNext) { Cache cache = (Cache)i$.next; ValueWrapper wrapper = cache.get(key); if (wrapper != null) { retVal = wrapper.get; cacheHit = true; break; } } } } } else { return null; } }

key = context.generateKey; 再跳转。

protected Object generateKey { if (StringUtils.hasText(this.operation.getKey)) { EvaluationContext evaluationContext = this.createEvaluationContext(ExpressionEvaluator.NO_RESULT); return CacheAspectSupport.this.evaluator.key(this.operation.getKey, this.method, evaluationContext); } else { return CacheAspectSupport.this.keyGenerator.generate(this.target, this.method, this.args); } }

到达getExpression方法,由于key在注解上面配置了,所以不为空,在继续跳转。

public Object key(String keyExpression, Method method, EvaluationContext evalContext) { return this.getExpression(this.keyCache, keyExpression, method).getValue(evalContext); } private Expression getExpression(Map cache, String expression, Method method) { String key = this.toString(method, expression); Expression rtn = (Expression)cache.get(key); if (rtn == null) { rtn = this.parser.parseExpression(expression); cache.put(key, rtn); } return rtn; }

最终来到了parser.parseExpression;

根据代码可以看到解析器用的是 private final SpelExpressionParser parser = new SpelExpressionParser;

可以得出结论就是Spel表达式这个东东吧。对于实体类+方法的表达式可能会实时去反射得到结果。那我们能不能再生产key的上层再加一层缓存呢?答案是肯定的。

我们可以通过javaPoet方式动态生成class的形式,将生成的类加载到内存中。通过它的实例来生成key。

javaPoet类似于javasis是一个用于动态生成代码的开源项目,通过这个类库下面的api我们来进行简易diy尝试。

上代码,忽略不重要部分,切面简写直接展示生成key的部分。

@Aspect@Componentpublic class CacheAspect { @Around("@annotation(myCache)") public Object around(ProceedingJoinPoint pjp, MyCache myCache) throws Throwable { long currentTime = System.currentTimeMillis; Object value = null; try { if(!myCache.useCache){ return pjp.proceed; } Object args = pjp.getArgs; if(args == null || args[0] == null){ return pjp.proceed; } Object obj = args[0]; String key = MyCacheCacheKeyGenerator.generatorCacheKey(myCache,obj.getClass.getDeclaredFields,obj); ...... } catch (Throwable throwable) { log.error("cache throwable",throwable); } return pjp.proceed; }}

缓存key生成接口。

public interface MyCacheKeyGenerator { /** * 生成key * */ String generateKey(Method method, Object args, Object target, String key);}

具体实现,其中wrapper是一个包装类,只是一个搬运工。通过key来动态产生key生成器。

public class DyCacheKeyGenerator implements MyCacheKeyGenerator { private final ConcurrentMap cacheMap = new ConcurrentHashMap; /** * 生成key * * @param method 调用的方法名字 * @param args 参数列表 * @param target 目标值 * @param key key的格式 * @return */ @Override public String generateKey(Method method, Object args, Object target, String key) { Wrapper wrapper = cacheMap.computeIfAbsent(key, k -> new Wrapper); getMykeyGenerator(method, key, wrapper); return ((MyCacheKeyGenerator) wrapper.getData).generate(args); } private void getMykeyGenerator(Method method, String key, Wrapper wrapper) { if (wrapper.getData != null) { return; } synchronized (wrapper) { if (wrapper.getData == null) { MyCacheKeyGenerator keyGenerator = MyCacheKeyGenerator.initMyKeyGenerator(method, key); wrapper.setData(keyGenerator); } } }}

那么我们首先根据key获取表达式的集合,如果是反射则会生成DynamicExpression表达式,连接符会生成静态的StaticExpression表达式。表达式持有了key中字符串的片段。

public static MyCacheKeyGenerator initMyKeyGenerator(Method method, String key) { Set importHashSet = new HashSet; //根据key中的配置的方法生成表达式列表 List expressionList = new LinkedList; generateExpression(key, expressionList); for (Expression expression : expressionList) { if (expression instanceof DynamicExpression) { String expressionStr = expression.execute; //判断格式合法性 String items = expressionStr.split("\\."); String indexValue = items[0].replace("args", ""); int index = Integer.parseInt(indexValue); Class clx = method.getParameterTypes[index]; importHashSet.add(clx); //获取对应属性的方法 String filedName = items[1]; String keyValue = Character.toUpperCase(filedName.charAt(0)) + filedName.substring(1); try { keyValue = "get" + keyValue; Method felidMethod = clx.getMethod(keyValue); expression.setExpression(String.format("String.valueOf(((%s)args[%s]).%s)", clx.getName, index, felidMethod.getName)); } catch (NoSuchMethodException e) { } } } // 定义接口类型 ClassName interfaceName = ClassName.get("com.xxx.xxx", "MyKeyGenerator"); // 定义类名和包名 ClassName className = ClassName.get("com.xxx.xxx", "DyMyKeyGeneratorImpl" + classIndex.incrementAndGet); // 创建类构造器 TypeSpec.Builder classBuilder = TypeSpec.classBuilder(className.simpleName) .addModifiers(Modifier.PUBLIC) .addSuperinterface(interfaceName); StringBuilder stringBuilder = new StringBuilder("stringBuilder"); for (Expression expression : expressionList) { stringBuilder.append(".append(").append(expression.execute).append(")"); } MethodSpec generateMethod = MethodSpec.methodBuilder("generate") .addModifiers(Modifier.PUBLIC) .returns(String.class) .addParameter(Object.class, "args") .addStatement("$T stringBuilder = new StringBuilder", StringBuilder.class) .addStatement(stringBuilder.toString) .addStatement("return $S", "stringBuilder.toString;") .build; classBuilder.addMethod(generateMethod); JavaFile javaFile = JavaFile.builder(className.packageName, classBuilder.build) .build; StringBuilder sb = new StringBuilder; try { javaFile.writeTo(sb); } catch (IOException e) { logger.error("写入StringBuilder失败", e); } try { System.out.println(sb.toString); Map results = compiler.compile(className + ".java", sb.toString); Class clazz = compiler.loadClass("com.xxx.xxx." + className, results); return (KeyGenerator) clazz.newInstance; } catch (Exception e) { logger.error("编译失败,编译内容:{}", sb.toString, e); throw new RuntimeException("内存class编译失败"); } } public static void generateExpression(String key, List expressionList) { if (StringUtils.isEmpty(key)) { return; } int index = key.indexOf(paramsPrefix); if (index

生成表达式列表后开始遍历,最终得到key中每个arg形参与对应的方法片段(key格式类似于@Cacheable 注解的用法。比如文章开始时候提到的我们可以改成这样使用,代码如下:)

@MyCache(key="#args0.getId"+"_"+"#args0.getName")public Person getByIsbn(Person person) { return personMapper.getPerson(person);}

将静态与动态片段重新拼接放入表达式中。然后我们使用JavaPoet的接口动态创建class,实现其中的generateKey方法,并且解析表达式填充到方法的实现中。最终将class加载到内存中,再生产一个实例,并将这个实例缓存到内存中。这样下次调用就可以使用动态生成的实例丝滑的拼接key啦!!

JavaPoet用法还有很多,而且@Cacheable还有很多灵活玩法,由于篇幅太长就不一一呈现了。respect!

来源:京东云开发者

相关推荐