jvm-sandbox-repeater 是阿里开源的一款可基于 jvm-sandbox (阿里另一开源项目)可对应用目标 jvm 进行动态增强同时对目标服务的指定流量进行录制及回放的工具,使用过程中遇到如下问题:
基于上述背景,我们打算对流量回放进行如下改造:
配置对象 RepeaterConfig 新增属性 List ignoreFiled 支持用户配置降噪字段。
/**
* 需要忽略对比的列信息
*/
private List<String> ignoreFiled = Lists.newArrayList();
对比器接口 com.alibaba.jvm.sandbox.repeater.aide.compare.Comparable#compare 新增入参 List ignoreFiled ,并对参数完成传递
/**
* compare to object
*
* @param left left object to be compare
* @param right right object to be compare
* @param ignoreFiled compare ignore some filed
* @return compare result
*/
CompareResult compare(Object left, Object right,List<String> ignoreFiled);
对比器 Map 对比器实现 com.alibaba.jvm.sandbox.repeater.aide.compare.comparator.MapComparator 进行改造,根据降噪字段进行忽略对比。
if (ignoreFiled != null && key instanceof String && ignoreFiled.contains(key)) {
continue;
}
结果对比,再保存回放结果处 com.alibaba.repeater.console.service.impl.ReplayServiceImpl#saveRepeat 进行改造。
List<String> ignoreFiledLists = null;
if (moduleConfigBORepeaterResult.getData() != null) {
ignoreFiledLists = moduleConfigBORepeaterResult.getData().getConfigModel().getIgnoreFiled();
}
CompareResult result = comparable.compare(actual, expect, ignoreFiledLists);
其他关联影响:由于对比器入参新增,导致所有调用对比器处都需要进行入参添加,不使用降噪配置的地方传 null 即可,比如
关键信息图解:
下面是一些基础代码:
public class AiDenoiseOfferJob extends QuartzJobBean {
@Override
protected void executeInternal(JobExecutionContext context) throws JobExecutionException {
}
}
public class AkkaConfig {
@Autowired
private ApplicationContext applicationContext;
@Autowired
private ApolloConfig apolloConfig;
@Bean
public ActorSystem actorSystem() {
return ActorSystem.create("my-actor-system");
}
@Bean
public ActorRef myActor(ActorSystem actorSystem) {
return actorSystem.actorOf(new RoundRobinPool(apolloConfig.getActorNum()).props(Props.create(MyActor.class,applicationContext)), "router");
}
}
@Override
protected void executeInternal(JobExecutionContext context) throws JobExecutionException {
//获取降噪流量详情
List<RecordBO> recordBOList = recordDenoise.getRecordDetailBOList(apolloConfig.getPageSize());
// 计算每个分区的大小
int partitionSize = (int) Math.ceil((double) recordBOList.size() / apolloConfig.getActorNum());
// 使用 IntStream 创建一个范围从 0 到recordBOList.size()-1 的流
IntStream.range(0, recordBOList.size())
.boxed()
.collect(Collectors.groupingBy(index -> index / partitionSize))
.values()
.stream()
.map(indices -> indices.stream().map(recordBOList::get).collect(Collectors.toList()))
.forEach(t -> {
// 向 MyActor 发送 MyMessage 消息,消息内容为 RecordBO 列表
myActorRef.tell(new MyMessage(t), ActorRef.noSender());
});
}
public class MyActor extends AbstractActor {
private final ApplicationContext applicationContext;
@Autowired
public MyActor(ApplicationContext applicationContext) {
this.applicationContext = applicationContext;
}
@Override
public Receive createReceive() {
return receiveBuilder()
.match(MyMessage.class, message -> {
List<RecordBO> recordBOList = message.getRecordBOList();
log.info("接受到的消息大小,{},当前线程:{}",recordBOList.size(),Thread.currentThread().getName());
RecordDenoise recordDenoise = applicationContext.getBean(RecordDenoise.class);
if (!recordBOList.isEmpty()) {
long startTime = System.currentTimeMillis();
//消费Actor消息
recordDenoise.doRecordDenoise(recordBOList);
long endTime = System.currentTimeMillis();
log.info("接受消息到消费完成耗时:{}",endTime-startTime);
}
})
.build();
}
}
private void processRecordDetailBO(RecordBO recordBO, List<RecordFlowEntity> idCosineSimilarity) {
//接口路径个性化处理
.....
//缓存处理的流量信息
....
//根据缓存流量id获取降噪纬度
...
//相似度计算
...
//根据相似度转用例
....
}
现阶段针对结果对比提供了手动降噪的能力,并将录制的流量进行了结果降噪对比同时将其转为用例,对于测试同学很难分辨这些用例覆盖了哪些代码,后续计划结合公司二开的基于 jacoco 的精准提供的能力对沉淀的用例进行训练,确保用户知晓用例对覆盖了哪些代码,甚至可能知道用例覆盖了哪些业务场景。目前正在调研中,尽请期待。