前往小程序,Get更优阅读体验!
立即前往
首页
学习
活动
专区
工具
TVP
发布
社区首页 >专栏 >聊聊SpinalTap的MysqlEventFilter

聊聊SpinalTap的MysqlEventFilter

作者头像
code4it
发布2020-06-03 10:25:25
2670
发布2020-06-03 10:25:25
举报
文章被收录于专栏:码匠的流水账

本文主要研究一下SpinalTap的MysqlEventFilter

MysqlEventFilter

SpinalTap/spinaltap-mysql/src/main/java/com/airbnb/spinaltap/mysql/event/filter/MysqlEventFilter.java

代码语言:javascript
复制
public abstract class MysqlEventFilter implements Filter<BinlogEvent> {
  public static Filter<BinlogEvent> create(
      @NonNull final TableCache tableCache,
      @NonNull final Set<String> tableNames,
      @NonNull final AtomicReference<SourceState> state) {
    return ChainedFilter.<BinlogEvent>builder()
        .addFilter(new EventTypeFilter())
        .addFilter(new TableFilter(tableCache, tableNames))
        .addFilter(new DuplicateFilter(state))
        .build();
  }
}
  • MysqlEventFilter提供了create方法,使用ChainedFilter来构造BinlogEvent的Filter,默认添加了EventTypeFilter、TableFilter、DuplicateFilter

EventTypeFilter

SpinalTap/spinaltap-mysql/src/main/java/com/airbnb/spinaltap/mysql/event/filter/EventTypeFilter.java

代码语言:javascript
复制
@RequiredArgsConstructor
final class EventTypeFilter extends MysqlEventFilter {
  @SuppressWarnings("unchecked")
  private static final Set<Class<? extends BinlogEvent>> WHITELISTED_EVENT_TYPES =
      ImmutableSet.of(
          TableMapEvent.class,
          WriteEvent.class,
          UpdateEvent.class,
          DeleteEvent.class,
          XidEvent.class,
          QueryEvent.class,
          StartEvent.class,
          GTIDEvent.class);

  public boolean apply(@NonNull final BinlogEvent event) {
    return WHITELISTED_EVENT_TYPES.contains(event.getClass());
  }
}
  • EventTypeFilter继承了MysqlEventFilter,其apply方法通过WHITELISTED_EVENT_TYPES来进行过滤

TableFilter

SpinalTap/spinaltap-mysql/src/main/java/com/airbnb/spinaltap/mysql/event/filter/TableFilter.java

代码语言:javascript
复制
@RequiredArgsConstructor
final class TableFilter extends MysqlEventFilter {
  @NonNull private final TableCache tableCache;
  @NonNull private final Set<String> tableNames;

  public boolean apply(@NonNull final BinlogEvent event) {
    if (event instanceof TableMapEvent) {
      TableMapEvent tableMap = (TableMapEvent) event;
      return tableNames.contains(
          Table.canonicalNameOf(tableMap.getDatabase(), tableMap.getTable()));
    } else if (event.isMutation()) {
      return tableCache.contains(event.getTableId());
    }

    return true;
  }
}
  • TableFilter继承了TableFilter,其apply方法在event为TableMapEvent是,通过database、table来判断,在event.isMutation()为true是通过tableCache.contains(event.getTableId())来判断,其余情况返回true

DuplicateFilter

SpinalTap/spinaltap-mysql/src/main/java/com/airbnb/spinaltap/mysql/event/filter/DuplicateFilter.java

代码语言:javascript
复制
@RequiredArgsConstructor
public final class DuplicateFilter extends MysqlEventFilter {
  @NonNull private final AtomicReference<SourceState> state;

  public boolean apply(@NonNull final BinlogEvent event) {
    // Only applies to mutation events
    if (!event.isMutation()) {
      return true;
    }

    // We need to tell if position in `event` and in `state` are from the same source
    // MySQL server, because a failover may have happened and we are currently streaming
    // from the new master.
    // If they are from the same source server, we can just use the binlog filename and
    // position (offset) to tell whether we should skip this event.
    BinlogFilePos eventBinlogPos = event.getBinlogFilePos();
    BinlogFilePos savedBinlogPos = state.get().getLastPosition();
    // Use the same logic in BinlogFilePos.compareTo() here...
    if (BinlogFilePos.shouldCompareUsingFilePosition(eventBinlogPos, savedBinlogPos)) {
      return event.getOffset() > state.get().getLastOffset();
    }

    // If this point is reached, a master failover might have happened.
    // We can only use GTIDSet to tell whether this event should be skipped.
    // We should only skip this event if GTIDSet in event is a "proper subset" of the GTIDSet
    // in saved state, because it is possible that the last transaction we streamed before the
    // failover is in the middle of a transaction.
    GtidSet eventGtidSet = eventBinlogPos.getGtidSet();
    GtidSet savedGtidSet = savedBinlogPos.getGtidSet();
    return !eventGtidSet.isContainedWithin(savedGtidSet) && !eventGtidSet.equals(savedGtidSet);
  }
}
  • DuplicateFilter继承了MysqlEventFilter,它会通过binlog的offset来判断

小结

MysqlEventFilter提供了create方法,使用ChainedFilter来构造BinlogEvent的Filter,默认添加了EventTypeFilter、TableFilter、DuplicateFilter

doc

  • MysqlEventFilter
本文参与 腾讯云自媒体同步曝光计划,分享自微信公众号。
原始发表:2020-05-31,如有侵权请联系 cloudcommunity@tencent.com 删除

本文分享自 码匠的流水账 微信公众号,前往查看

如有侵权,请联系 cloudcommunity@tencent.com 删除。

本文参与 腾讯云自媒体同步曝光计划  ,欢迎热爱写作的你一起参与!

评论
登录后参与评论
0 条评论
热度
最新
推荐阅读
目录
  • MysqlEventFilter
  • EventTypeFilter
  • TableFilter
  • DuplicateFilter
  • 小结
  • doc
相关产品与服务
云数据库 SQL Server
腾讯云数据库 SQL Server (TencentDB for SQL Server)是业界最常用的商用数据库之一,对基于 Windows 架构的应用程序具有完美的支持。TencentDB for SQL Server 拥有微软正版授权,可持续为用户提供最新的功能,避免未授权使用软件的风险。具有即开即用、稳定可靠、安全运行、弹性扩缩等特点。
领券
问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档