副本跨路径迁移

经常有小伙伴有这样的疑问：为什么线上Kafka机器各个磁盘间的占用不均匀，经常出现“一边倒”的情形？ 这是因为Kafka只保证分区数量在各个磁盘上均匀分布，但它无法知晓每个分区实际占用空间，故很有可能出现某些分区消息数量巨大导致占用大量磁盘空间的情况。在1.1版本之前，用户对此毫无办法，因为1.1之前Kafka只支持分区数据在不同broker间的重分配，而无法做到在同一个broker下的不同磁盘间做重分配。1.1版本正式支持副本在不同路径间的迁移，具体的实现细节详见KIP-113。本文简单演示一下该新功能的用法。

假设我在Kafka broker的server.properties文件中配置了多个路径(代表多块磁盘)，如下所示：

...

############################# Log Basics #############################

# A comma seperated list of directories under which to store log files

log.dirs=/Users/huxi/SourceCode/newenv/datalogs/kafka_1,/Users/huxi/SourceCode/newenv/datalogs/kafka_2,/Users/huxi/SourceCode/newenv/datalogs/kafka_3

...

之后我创建了一个9分区的topic，并发送了9百万条消息。查询这些目录发现Kafka均匀地将9个分区分布到这三个路径上，如下所示：

ll kafka_1/ |grep test-topic

drwxr-xr-x6 huxistaff192 Jun 22 17:21 test-topic-3

drwxr-xr-x6 huxistaff192 Jun 22 17:21 test-topic-4

drwxr-xr-x6 huxistaff192 Jun 22 17:21 test-topic-5

ll kafka_2/ |grep test-topic

drwxr-xr-x6 huxistaff192 Jun 22 17:21 test-topic-0

drwxr-xr-x6 huxistaff192 Jun 22 17:21 test-topic-1

drwxr-xr-x6 huxistaff192 Jun 22 17:21 test-topic-2

ll kafka_3/ |grep test-topic

drwxr-xr-x6 huxistaff192 Jun 22 17:21 test-topic-6

drwxr-xr-x6 huxistaff192 Jun 22 17:21 test-topic-7

drwxr-xr-x6 huxistaff192 Jun 22 17:21 test-topic-8

现在我们想要将test-topic的6，7，8分区全部迁移到kafka_2路径下，并且把test-topic的1分区迁移到kafka_1下。若要实现这个需求，我们首先需要编写一个JSON文件，假定名为migrate-replica.json:

{"partitions":[{"topic": "test-topic","partition": 1,"replicas": [0],"log_dirs": ["/Users/huxi/SourceCode/newenv/datalogs/kafka_1"]},{"topic": "test-topic","partition": 6,"replicas": [0],"log_dirs": ["/Users/huxi/SourceCode/newenv/datalogs/kafka_2"]},{"topic": "test-topic","partition": 7,"replicas": [0],"log_dirs": ["/Users/huxi/SourceCode/newenv/datalogs/kafka_2"]},{"topic": "test-topic","partition": 8,"replicas": [0],"log_dirs": ["/Users/huxi/SourceCode/newenv/datalogs/kafka_2"]}],"version":1}

其中，replicas中的0表示broker ID，由于本文只启动了一个broker，且broker.id = 0，故这里只写0即可。实际上你可以指定多个broker实现为多个broker同时迁移副本的功能。另外当前的version固定是1.

保存好这个JSON后，我们执行以下命令执行副本迁移：

bin/kafka-reassign-partitions.sh--zookeeper localhost:2181 --bootstrap-server localhost:9092 --reassignment-json-file ../migrate-replica.json --execute

Current partition replica assignment

{"version":1,"partitions":[{"topic":"test-topic","partition":8,"replicas":[0],"log_dirs":["any"]},{"topic":"test-topic","partition":4,"replicas":[0],"log_dirs":["any"]},{"topic":"test-topic","partition":5,"replicas":[0],"log_dirs":["any"]},{"topic":"test-topic","partition":2,"replicas":[0],"log_dirs":["any"]},{"topic":"test-topic","partition":6,"replicas":[0],"log_dirs":["any"]},{"topic":"test-topic","partition":3,"replicas":[0],"log_dirs":["any"]},{"topic":"test-topic","partition":1,"replicas":[0],"log_dirs":["any"]},{"topic":"test-topic","partition":7,"replicas":[0],"log_dirs":["any"]},{"topic":"test-topic","partition":0,"replicas":[0],"log_dirs":["any"]}]}

Save this to use as the --reassignment-json-file option during rollback

Successfully started reassignment of partitions.

再次查看路径副本分布：

ll kafka_1/ |grep test-topic

drwxr-xr-x6 huxistaff192 Jun 22 17:31 test-topic-1

drwxr-xr-x6 huxistaff192 Jun 22 17:21 test-topic-3

drwxr-xr-x6 huxistaff192 Jun 22 17:21 test-topic-4

drwxr-xr-x6 huxistaff192 Jun 22 17:21 test-topic-5

ll kafka_2/ |grep test-topic

drwxr-xr-x6 huxistaff192 Jun 22 17:21 test-topic-0

drwxr-xr-x6 huxistaff192 Jun 22 17:21 test-topic-2

drwxr-xr-x6 huxistaff192 Jun 22 17:31 test-topic-6

drwxr-xr-x6 huxistaff192 Jun 22 17:31 test-topic-7

drwxr-xr-x6 huxistaff192 Jun 22 17:31 test-topic-8

ll kafka_3/ |grep test-topic

显然，6，7，8已经被成功地迁移到kafka_2下，而分区1也迁移到了kafka_1下。值得一提的是，不仅所有的日志段、索引文件被迁移，实际上分区外层的checkpoint文件也会被更新。比如我们检查kafka_2下的replication-offset-checkpoint文件可以发现，现在该文件已经包含了6，7，8分区的位移数据，如下所示：

cat replication-offset-checkpoint

7

test-topic 8 1000000

test-topic 2 1000000

test 0 1285714

test-topic 6 1000000

test-topic 7 1000000

test-topic 0 1000000

test 2 1285714

以上就是对1.1新功能“副本跨路径迁移”的简单尝试，希望对有此困扰的用户有用~~