Redis 3.0 Cluster配置文档

V2.0

   

说明：文档如有纰漏，欢迎提出指导建议，​​​​

文档控制


版本



内容



时间



作者



V1.0



初始版本



2015/4/23



张耐



V2.0



修复部分文字描述不准确问题



 2015/5/10



 张耐


 

目录

​​准备阶段​​

​​安装Cluster​​

​​管理cluster​​

​​（1）       添加master节点​​

​​（2）       数据分片​​

​​（3）       删除节点​​

 

 Redis 3.0概述

Redis 是一个高性能的key-value数据库。 redis的出现，很大程度补偿了​​memcached​​这类keyvalue存储的不足，在部分场合可以对关系数据库起到很好的补充作用。它提供了Python，Ruby，Erlang，PHP客户端，使用很方便。3.0版本加入cluster功能，解决了redis单点无法横向扩展问题。

架构拓扑

3.0版本最大的特点就是支持cluster分布式横向扩展，下面为3个Master节点以及3个slave节点的拓扑图：

 ​Redis集群架构部署（Version <wbr>3.0.7)​


APP1


M1，M2，M3为redis三个主节点，S1，S2，S3为redis三个从节点，分别为M1，M2，M3备份数据以及故障切换使用。APP访问数据库可以通过连接任意一个Master节点实现。在三个Master节点的redis集群中，只容许有一个Master出故障，当多于一个Master宕机时，redis即不可用。当其中一个Master出现故障，其对应的Slave会接管故障Master的服务，保证redis 数据库的正常使用。

准备阶段

（1）       安装包

​​ http://redis.io​​

（2）       系统包

安装gcc：yum install gcc

安装zlib：yum install zib

安装ruby：yum install ruby

安装rubygems：yum install rubygems

安装gem redis：（下载：​​http://rubygems.org/gems/redis/versions/3.0.7​​）

# gem install -l /tmp/redis-3.0.7.gem

Successfully installed redis-3.0.7

1 gem installed

Installing ri documentation for redis-3.0.7...

Installing RDoc documentation for redis-3.0.7...

（3）       系统参数

修改open files：# ulimit -n  10032 （默认1024）

添加vm.overcommit_memory=1：

#vi /etc/sysctl.conf

#sysctl vm.overcommit_memory=1

关闭hugepage：# echo never > /sys/kernel/mm/transparent_hugepage/enabled

修改somaxconn ：# echo 511 >/proc/sys/net/core/somaxconn

关闭防火墙：# service iptables stop

关闭selinux：# vi /etc/sysconfig/selinux  修改“SELINUX=disabled”

 安装Cluster

（1）       安装软件

# cd /redis/redis-3.0.0

# make

# make install

 

（2）       拷贝bin文件

# cp /redis/redis-3.0.0/src/redis-trib.rb /usr/local/bin/

# cp redis-cli /usr/local/bin/

# cp redis-server /usr/local/bin/

# which redis-trib.rb

/usr/local/bin/redis-trib.rb

 

（3）       配置通用config文件redis-common.conf

# vi /redis/redis-3.0.0/config/redis-common.conf


代码如下：



#GENERAL 

daemonize yes 

tcp-backlog 511 

timeout 0 

tcp-keepalive 0 

loglevel notice 

databases 16 

dir /redis/redis-3.0.0/data 

slave-serve-stale-data yes 

slave-read-only yes 

#not use default 

repl-disable-tcp-nodelay yes 

slave-priority 100 

appendonly yes 

appendfsync everysec 

no-appendfsync-on-rewrite yes 

auto-aof-rewrite-min-size 64mb 

lua-time-limit 5000 

cluster-enabled yes 

cluster-node-timeout 15000 

cluster-migration-barrier 1 

slowlog-log-slower-than 10000 

slowlog-max-len 128 

notify-keyspace-events "" 

hash-max-ziplist-entries 512 

hash-max-ziplist-value 64 

list-max-ziplist-entries 512 

list-max-ziplist-value 64 

set-max-intset-entries 512 

zset-max-ziplist-entries 128 

zset-max-ziplist-value 64 

activerehashing yes 

client-output-buffer-limit normal 0 0 0 

client-output-buffer-limit slave 256mb 64mb 60 

client-output-buffer-limit pubsub 32mb 8mb 60 

hz 10 

aof-rewrite-incremental-fsync yes


 

（4）       节点1配置文件redis-6379.conf

# vi /redis/redis-3.0.0/config/redis-6379.conf


代码如下：



include /redis/redis-3.0.0/config/redis-common.conf 

port 6379 

logfile "/redis/redis-3.0.0/log/redis-6379.log"

maxmemory 100m 

# volatile-lru -> remove the key with an expire set using an LRU algorithm 

# allkeys-lru -> remove any key accordingly to the LRU algorithm 

# volatile-random -> remove a random key with an expire set 

# allkeys-random -> remove a random key, any key 

# volatile-ttl -> remove the key with the nearest expire time (minor TTL) 

# noeviction -> don't expire at all, just return an error on write operations 

maxmemory-policy allkeys-lru 

appendfilename "appendonly-6379.aof" 

dbfilename dump-6379.rdb

#dir /redis/redis-3.0.0/data

cluster-config-file nodes-6379.conf 

auto-aof-rewrite-percentage 80-100


 

 

（5）       节点2配置文件redis-6389.conf

# vi /redis/redis-3.0.0/config/redis-6389.conf


代码如下：



include /redis/redis-3.0.0/config/redis-common.conf 

port 6389 

logfile "/redis/redis-3.0.0/log/redis-6389.log"

maxmemory 100m 

# volatile-lru -> remove the key with an expire set using an LRU algorithm 

# allkeys-lru -> remove any key accordingly to the LRU algorithm 

# volatile-random -> remove a random key with an expire set 

# allkeys-random -> remove a random key, any key 

# volatile-ttl -> remove the key with the nearest expire time (minor TTL) 

# noeviction -> don't expire at all, just return an error on write operations 

maxmemory-policy allkeys-lru 

appendfilename "appendonly-6389.aof" 

dbfilename dump-6389.rdb

cluster-config-file nodes-6389.conf 

auto-aof-rewrite-percentage 80-100


 

 

（6）       节点3配置文件redis-6399.conf

# vi /redis/redis-3.0.0/config/redis-6399.conf


代码如下：



include /redis/redis-3.0.0/config/redis-common.conf 

port 6399 

logfile "/redis/redis-3.0.0/log/redis-6399.log"

maxmemory 100m 

# volatile-lru -> remove the key with an expire set using an LRU algorithm 

# allkeys-lru -> remove any key accordingly to the LRU algorithm 

# volatile-random -> remove a random key with an expire set 

# allkeys-random -> remove a random key, any key 

# volatile-ttl -> remove the key with the nearest expire time (minor TTL) 

# noeviction -> don't expire at all, just return an error on write operations 

maxmemory-policy allkeys-lru 

appendfilename "appendonly-6399.aof"  

dbfilename dump-6399.rdb

cluster-config-file nodes-6399.conf 

auto-aof-rewrite-percentage 80-100


 

 

（7）       启动redis节点

# redis-server redis-6379.conf

# redis-server redis-6389.conf

# redis-server redis-6399.conf

# redis-server redis-7379.conf

# redis-server redis-7389.conf

# redis-server redis-7399.conf

 

# ps -ef| grep redis

root      4623     1  0 11:07 ?        00:00:00 redis-server *:6379 [cluster]

root      4627     1  0 11:07 ?        00:00:00 redis-server *:6389 [cluster]

root      4631     1  0 11:07 ?        00:00:00 redis-server *:6399 [cluster]

root      4654     1  0 11:30 ?        00:00:00 redis-server *:7379 [cluster]

root      4658     1  0 11:30 ?        00:00:00 redis-server *:7389 [cluster]

root      4662     1  0 11:30 ?        00:00:00 redis-server *:7399 [cluster]

 

（8）       通过redis-trib创建cluster

#--replicas 则指定了为Redis Cluster中的每个Master节点配备几个Slave节点

# redis-trib.rb create --replicas 1 192.168.3.88:6379 192.168.3.88:6389 192.168.3.88:6399 192.168.3.88:7379 192.168.3.88:7389 192.168.3.88:7399

>>> Creating cluster

Connecting to node 192.168.3.88:6379: OK

Connecting to node 192.168.3.88:6389: OK

Connecting to node 192.168.3.88:6399: OK

Connecting to node 192.168.3.88:7379: OK

Connecting to node 192.168.3.88:7389: OK

Connecting to node 192.168.3.88:7399: OK

>>> Performing hash slots allocation on 6 nodes...

Using 3 masters:

192.168.3.88:6379

192.168.3.88:6389

192.168.3.88:6399

Adding replica 192.168.3.88:7379 to 192.168.3.88:6379

Adding replica 192.168.3.88:7389 to 192.168.3.88:6389

Adding replica 192.168.3.88:7399 to 192.168.3.88:6399

M: 05fe758161e2cbe23240697f47f1cd2c937a675b 192.168.3.88:6379

   slots:0-5460 (5461 slots) master

M: d1d124d35c848e9c8e726b59af669c9196557869 192.168.3.88:6389

   slots:5461-10922 (5462 slots) master

M: d64223d6695fcc7e1030f219f09d7488c438cf39 192.168.3.88:6399

   slots:10923-16383 (5461 slots) master

S: 7f77ec03e40d0cc9f343d783a293ae8aa6c6e090 192.168.3.88:7379

   replicates 05fe758161e2cbe23240697f47f1cd2c937a675b

S: 98dae5126228dea54d1321eeb357d8773bd2ee11 192.168.3.88:7389

   replicates d1d124d35c848e9c8e726b59af669c9196557869

S: d013aee7cae8163f787cb6445778ff97bf66ce17 192.168.3.88:7399

   replicates d64223d6695fcc7e1030f219f09d7488c438cf39

Can I set the above configuration? (type 'yes' to accept): yes

>>> Nodes configuration updated

>>> Assign a different config epoch to each node

>>> Sending CLUSTER MEET messages to join the cluster

Waiting for the cluster to join......

>>> Performing Cluster Check (using node 192.168.3.88:6379)

M: 05fe758161e2cbe23240697f47f1cd2c937a675b 192.168.3.88:6379

   slots:0-5460 (5461 slots) master

M: d1d124d35c848e9c8e726b59af669c9196557869 192.168.3.88:6389

   slots:5461-10922 (5462 slots) master

M: d64223d6695fcc7e1030f219f09d7488c438cf39 192.168.3.88:6399

   slots:10923-16383 (5461 slots) master

M: 7f77ec03e40d0cc9f343d783a293ae8aa6c6e090 192.168.3.88:7379

   slots: (0 slots) master

   replicates 05fe758161e2cbe23240697f47f1cd2c937a675b

M: 98dae5126228dea54d1321eeb357d8773bd2ee11 192.168.3.88:7389

   slots: (0 slots) master

   replicates d1d124d35c848e9c8e726b59af669c9196557869

M: d013aee7cae8163f787cb6445778ff97bf66ce17 192.168.3.88:7399

   slots: (0 slots) master

   replicates d64223d6695fcc7e1030f219f09d7488c438cf39

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[OK] All 16384 slots covered.

 

（9）       检查集群状态

连接任意节点，执行redis-trib.rb

# redis-trib.rb check 192.168.3.88:6379

Connecting to node 192.168.3.88:6379: OK

Connecting to node 192.168.3.88:7389: OK

Connecting to node 192.168.3.88:7399: OK

Connecting to node 192.168.3.88:6389: OK

Connecting to node 192.168.3.88:6399: OK

Connecting to node 192.168.3.88:7379: OK

>>> Performing Cluster Check (using node 192.168.3.88:6379)

M: 05fe758161e2cbe23240697f47f1cd2c937a675b 192.168.3.88:6379

   slots:0-5460 (5461 slots) master

   1 additional replica(s)

S: 98dae5126228dea54d1321eeb357d8773bd2ee11 192.168.3.88:7389

   slots: (0 slots) slave

   replicates d1d124d35c848e9c8e726b59af669c9196557869

S: d013aee7cae8163f787cb6445778ff97bf66ce17 192.168.3.88:7399

   slots: (0 slots) slave

   replicates d64223d6695fcc7e1030f219f09d7488c438cf39

M: d1d124d35c848e9c8e726b59af669c9196557869 192.168.3.88:6389

   slots:5461-10922 (5462 slots) master

   1 additional replica(s)

M: d64223d6695fcc7e1030f219f09d7488c438cf39 192.168.3.88:6399

   slots:10923-16383 (5461 slots) master

   1 additional replica(s)

S: 7f77ec03e40d0cc9f343d783a293ae8aa6c6e090 192.168.3.88:7379

   slots: (0 slots) slave

   replicates 05fe758161e2cbe23240697f47f1cd2c937a675b

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[OK] All 16384 slots covered.

 

管理cluster

（1）       添加master节点

添加的master节点配置在另一个服务器上，首先配置config文件

# vi /redis/redis-3.0.0/config/redis-6379.conf

# vi /redis/redis-3.0.0/config/redis-7379.conf

 

使用redis-trib.rb添加节点

在已有集群服务器（192.168.3.88）上执行

（注意：add-node的使用方法为new_host:new_port existing_host:existing_port，前面是新添加的节点信息，后面是已存在的节点信息）

# redis-trib.rb add-node 192.168.3.61:6379 192.168.3.88:6379

>>> Adding node 192.168.3.61:6379 to cluster 192.168.3.88:6379

Connecting to node 192.168.3.88:6379: OK

Connecting to node 192.168.3.88:7389: OK

Connecting to node 192.168.3.88:7399: OK

Connecting to node 192.168.3.88:6389: OK

Connecting to node 192.168.3.88:6399: OK

Connecting to node 192.168.3.88:7379: OK

>>> Performing Cluster Check (using node 192.168.3.88:6379)

M: 05fe758161e2cbe23240697f47f1cd2c937a675b 192.168.3.88:6379

   slots:0-5460 (5461 slots) master

   1 additional replica(s)

S: 98dae5126228dea54d1321eeb357d8773bd2ee11 192.168.3.88:7389

   slots: (0 slots) slave

   replicates d1d124d35c848e9c8e726b59af669c9196557869

S: d013aee7cae8163f787cb6445778ff97bf66ce17 192.168.3.88:7399

   slots: (0 slots) slave

   replicates d64223d6695fcc7e1030f219f09d7488c438cf39

M: d1d124d35c848e9c8e726b59af669c9196557869 192.168.3.88:6389

   slots:5461-10922 (5462 slots) master

   1 additional replica(s)

M: d64223d6695fcc7e1030f219f09d7488c438cf39 192.168.3.88:6399

   slots:10923-16383 (5461 slots) master

   1 additional replica(s)

S: 7f77ec03e40d0cc9f343d783a293ae8aa6c6e090 192.168.3.88:7379

   slots: (0 slots) slave

   replicates 05fe758161e2cbe23240697f47f1cd2c937a675b

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[OK] All 16384 slots covered.

Connecting to node 192.168.3.61:6379: OK

>>> Send CLUSTER MEET to node 192.168.3.61:6379 to make it join the cluster.

[OK] New node added correctly.

 

选择其中一个节点，检查集群状态，发现刚添加的节点已经在集群中了，角色是master，但是并没有slot分配到新加的节点上，后面要通过shard命令分配slot。

# redis-trib.rb check 192.168.3.88:6379

Connecting to node 192.168.3.88:6379: OK

Connecting to node 192.168.3.88:7389: OK

Connecting to node 192.168.3.88:7399: OK

Connecting to node 192.168.3.88:6389: OK

Connecting to node 192.168.3.61:6379: OK

Connecting to node 192.168.3.88:6399: OK

Connecting to node 192.168.3.88:7379: OK

>>> Performing Cluster Check (using node 192.168.3.88:6379)

M: 05fe758161e2cbe23240697f47f1cd2c937a675b 192.168.3.88:6379

   slots:0-5460 (5461 slots) master

   1 additional replica(s)

S: 98dae5126228dea54d1321eeb357d8773bd2ee11 192.168.3.88:7389

   slots: (0 slots) slave

   replicates d1d124d35c848e9c8e726b59af669c9196557869

S: d013aee7cae8163f787cb6445778ff97bf66ce17 192.168.3.88:7399

   slots: (0 slots) slave

   replicates d64223d6695fcc7e1030f219f09d7488c438cf39

M: d1d124d35c848e9c8e726b59af669c9196557869 192.168.3.88:6389

   slots:5461-10922 (5462 slots) master

   1 additional replica(s)

M: 89be535ff56586dcec56f14122add80d89a57bb3 192.168.3.61:6379

   slots: (0 slots) master

   0 additional replica(s)

M: d64223d6695fcc7e1030f219f09d7488c438cf39 192.168.3.88:6399

   slots:10923-16383 (5461 slots) master

   1 additional replica(s)

S: 7f77ec03e40d0cc9f343d783a293ae8aa6c6e090 192.168.3.88:7379

   slots: (0 slots) slave

   replicates 05fe758161e2cbe23240697f47f1cd2c937a675b

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[OK] All 16384 slots covered.

 

主节点添加完毕后，需要给该主节点添加一个slave节点，添加过程和主节点一直，添加完成后需要在redis中进行设置。

# redis-trib.rb add-node 192.168.3.61:7379 192.168.3.88:6379

显示内容省略

 

链接到要添加的slave数据库中，执行replicate操作。后面的ID为Master 192.168.3.61:6379的ID，通过redis-trib.rb check可以看到。

# redis-cli -c -h 192.168.3.61 -p 7379

192.168.3.61:7379> cluster replicate 89be535ff56586dcec56f14122add80d89a57bb3

OK

 

根据check结果，可以看到新添加的slave以及成功和Master建立联系。

# redis-trib.rb check 192.168.3.88:6379

Connecting to node 192.168.3.88:6379: OK

Connecting to node 192.168.3.88:7389: OK

Connecting to node 192.168.3.88:7399: OK

Connecting to node 192.168.3.88:6389: OK

Connecting to node 192.168.3.61:6379: OK

Connecting to node 192.168.3.61:7379: OK

Connecting to node 192.168.3.88:6399: OK

Connecting to node 192.168.3.88:7379: OK

>>> Performing Cluster Check (using node 192.168.3.88:6379)

M: 05fe758161e2cbe23240697f47f1cd2c937a675b 192.168.3.88:6379

   slots:0-5460 (5461 slots) master

   1 additional replica(s)

S: 98dae5126228dea54d1321eeb357d8773bd2ee11 192.168.3.88:7389

   slots: (0 slots) slave

   replicates d1d124d35c848e9c8e726b59af669c9196557869

S: d013aee7cae8163f787cb6445778ff97bf66ce17 192.168.3.88:7399

   slots: (0 slots) slave

   replicates d64223d6695fcc7e1030f219f09d7488c438cf39

M: d1d124d35c848e9c8e726b59af669c9196557869 192.168.3.88:6389

   slots:5461-10922 (5462 slots) master

   1 additional replica(s)

M: 89be535ff56586dcec56f14122add80d89a57bb3 192.168.3.61:6379

   slots: (0 slots) master

   1 additional replica(s)

S: 92017f0258675b02a7799726339efabf7d005f8c 192.168.3.61:7379

   slots: (0 slots) slave

   replicates 89be535ff56586dcec56f14122add80d89a57bb3

M: d64223d6695fcc7e1030f219f09d7488c438cf39 192.168.3.88:6399

   slots:10923-16383 (5461 slots) master

   1 additional replica(s)

S: 7f77ec03e40d0cc9f343d783a293ae8aa6c6e090 192.168.3.88:7379

   slots: (0 slots) slave

   replicates 05fe758161e2cbe23240697f47f1cd2c937a675b

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[OK] All 16384 slots covered.

 

（2）         数据分片

加入新的节点后，需要将其他的节点中的hash slot移动到新的节点中，以达到负载均衡的效果，指定集群中其中一个节点的地址

# redis-trib.rb reshard 192.168.3.6379

Connecting to node 192.168.3.88:6379: OK

Connecting to node 192.168.3.88:7389: OK

Connecting to node 192.168.3.88:7399: OK

Connecting to node 192.168.3.88:6389: OK

Connecting to node 192.168.3.61:6379: OK

Connecting to node 192.168.3.61:7379: OK

Connecting to node 192.168.3.88:6399: OK

Connecting to node 192.168.3.88:7379: OK

>>> Performing Cluster Check (using node 192.168.3.88:6379)

M: 05fe758161e2cbe23240697f47f1cd2c937a675b 192.168.3.88:6379

   slots:0-5460 (5461 slots) master

   1 additional replica(s)

S: 98dae5126228dea54d1321eeb357d8773bd2ee11 192.168.3.88:7389

   slots: (0 slots) slave

   replicates d1d124d35c848e9c8e726b59af669c9196557869

S: d013aee7cae8163f787cb6445778ff97bf66ce17 192.168.3.88:7399

   slots: (0 slots) slave

   replicates d64223d6695fcc7e1030f219f09d7488c438cf39

M: d1d124d35c848e9c8e726b59af669c9196557869 192.168.3.88:6389

   slots:5461-10922 (5462 slots) master

   1 additional replica(s)

M: 89be535ff56586dcec56f14122add80d89a57bb3 192.168.3.61:6379

   slots: (0 slots) master

   1 additional replica(s)

S: 92017f0258675b02a7799726339efabf7d005f8c 192.168.3.61:7379

   slots: (0 slots) slave

   replicates 89be535ff56586dcec56f14122add80d89a57bb3

M: d64223d6695fcc7e1030f219f09d7488c438cf39 192.168.3.88:6399

   slots:10923-16383 (5461 slots) master

   1 additional replica(s)

S: 7f77ec03e40d0cc9f343d783a293ae8aa6c6e090 192.168.3.88:7379

   slots: (0 slots) slave

   replicates 05fe758161e2cbe23240697f47f1cd2c937a675b

 [OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[OK] All 16384 slots covered.

How many slots do you want to move (from 1 to 16384)?4096  --16384/4=4096.master建议为基数

What is the receiving node ID? 89be535ff56586dcec56f14122add80d89a57bb3  --新加的主节点ID

Please enter all the source node IDs.

  Type 'all' to use all the nodes as source nodes for the hash slots.

  Type 'done' once you entered all the source nodes IDs.

Source node #1:all  --从所有其他master节点均匀把slot移动到新加的主节点

  .....

  .....

Moving slot 12284 from 192.168.3.88:6399 to 192.168.3.61:6379:

Moving slot 12285 from 192.168.3.88:6399 to 192.168.3.61:6379:

Moving slot 12286 from 192.168.3.88:6399 to 192.168.3.61:6379:

Moving slot 12287 from 192.168.3.88:6399 to 192.168.3.61:6379:

 

再次check，发现所有主节点的slot都变成4096了

# redis-trib.rb check 192.168.3.88:6379

Connecting to node 192.168.3.88:6379: OK

Connecting to node 192.168.3.88:7389: OK

Connecting to node 192.168.3.88:7399: OK

Connecting to node 192.168.3.88:6389: OK

Connecting to node 192.168.3.61:6379: OK

Connecting to node 192.168.3.61:7379: OK

Connecting to node 192.168.3.88:6399: OK

Connecting to node 192.168.3.88:7379: OK

>>> Performing Cluster Check (using node 192.168.3.88:6379)

M: 05fe758161e2cbe23240697f47f1cd2c937a675b 192.168.3.88:6379

   slots:1365-5460 (4096 slots) master

   1 additional replica(s)

S: 98dae5126228dea54d1321eeb357d8773bd2ee11 192.168.3.88:7389

   slots: (0 slots) slave

   replicates d1d124d35c848e9c8e726b59af669c9196557869

S: d013aee7cae8163f787cb6445778ff97bf66ce17 192.168.3.88:7399

   slots: (0 slots) slave

   replicates d64223d6695fcc7e1030f219f09d7488c438cf39

M: d1d124d35c848e9c8e726b59af669c9196557869 192.168.3.88:6389

   slots:6827-10922 (4096 slots) master

   1 additional replica(s)

M: 89be535ff56586dcec56f14122add80d89a57bb3 192.168.3.61:6379

   slots:0-1364,5461-6826,10923-12287 (4096 slots) master

   1 additional replica(s)

S: 92017f0258675b02a7799726339efabf7d005f8c 192.168.3.61:7379

   slots: (0 slots) slave

   replicates 89be535ff56586dcec56f14122add80d89a57bb3

M: d64223d6695fcc7e1030f219f09d7488c438cf39 192.168.3.88:6399

   slots:12288-16383 (4096 slots) master

   1 additional replica(s)

S: 7f77ec03e40d0cc9f343d783a293ae8aa6c6e090 192.168.3.88:7379

   slots: (0 slots) slave

   replicates 05fe758161e2cbe23240697f47f1cd2c937a675b

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[OK] All 16384 slots covered.

 

（3）         删除节点

删除主节点之前，需要先将slot迁移到其他主节点上

# redis-trib.rb reshard 192.168.3.88:6379

Connecting to node 192.168.3.88:6379: OK

Connecting to node 192.168.3.88:7389: OK

Connecting to node 192.168.3.88:7399: OK

Connecting to node 192.168.3.88:6389: OK

Connecting to node 192.168.3.61:6379: OK

Connecting to node 192.168.3.61:7379: OK

Connecting to node 192.168.3.88:6399: OK

Connecting to node 192.168.3.88:7379: OK

>>> Performing Cluster Check (using node 192.168.3.88:6379)

M: 05fe758161e2cbe23240697f47f1cd2c937a675b 192.168.3.88:6379

   slots:1365-5460,12288-16383 (8192 slots) master

   1 additional replica(s)

S: 98dae5126228dea54d1321eeb357d8773bd2ee11 192.168.3.88:7389

   slots: (0 slots) slave

   replicates d1d124d35c848e9c8e726b59af669c9196557869

S: d013aee7cae8163f787cb6445778ff97bf66ce17 192.168.3.88:7399

   slots: (0 slots) slave

   replicates d64223d6695fcc7e1030f219f09d7488c438cf39

M: d1d124d35c848e9c8e726b59af669c9196557869 192.168.3.88:6389

   slots:6827-10922 (4096 slots) master

   1 additional replica(s)

M: 89be535ff56586dcec56f14122add80d89a57bb3 192.168.3.61:6379

   slots:0-1364,5461-6826,10923-12287 (4096 slots) master

   1 additional replica(s)

S: 92017f0258675b02a7799726339efabf7d005f8c 192.168.3.61:7379

   slots: (0 slots) slave

   replicates 89be535ff56586dcec56f14122add80d89a57bb3

M: d64223d6695fcc7e1030f219f09d7488c438cf39 192.168.3.88:6399

   slots: (0 slots) master

   1 additional replica(s)

S: 7f77ec03e40d0cc9f343d783a293ae8aa6c6e090 192.168.3.88:7379

   slots: (0 slots) slave

   replicates 05fe758161e2cbe23240697f47f1cd2c937a675b

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[OK] All 16384 slots covered.

How many slots do you want to move (from 1 to 16384)? 4906

What is the receiving node ID? 89be535ff56586dcec56f14122add80d89a57bb3

Please enter all the source node IDs.

  Type 'all' to use all the nodes as source nodes for the hash slots.

  Type 'done' once you entered all the source nodes IDs.

Source node #1:d1d124d35c848e9c8e726b59af669c9196557869

Source node #2:done

……

……

    Moving slot 10920 from d1d124d35c848e9c8e726b59af669c9196557869

    Moving slot 10921 from d1d124d35c848e9c8e726b59af669c9196557869

    Moving slot 10922 from d1d124d35c848e9c8e726b59af669c9196557869

Do you want to proceed with the proposed reshard plan (yes/no)?yes

……

……

Moving slot 10920 from 192.168.3.88:6389 to 192.168.3.61:6379:

Moving slot 10921 from 192.168.3.88:6389 to 192.168.3.61:6379:

Moving slot 10922 from 192.168.3.88:6389 to 192.168.3.61:6379:

 

检查节点的slot是否完全迁移走，完成后就可以删除节点了

# redis-trib.rb check 192.168.3.88:6399

# redis-trib.rb del-node 192.168.3.88:6399 d64223d6695fcc7e1030f219f09d7488c438cf39

>>> Removing node d64223d6695fcc7e1030f219f09d7488c438cf39 from cluster 192.168.3.88:6399

Connecting to node 192.168.3.88:6399: OK

Connecting to node 192.168.3.61:6379: OK

Connecting to node 192.168.3.88:6379: OK

Connecting to node 192.168.3.88:7389: OK

Connecting to node 192.168.3.88:7379: OK

Connecting to node 192.168.3.88:6389: OK

Connecting to node 192.168.3.61:7379: OK

Connecting to node 192.168.3.88:7399: OK

>>> Sending CLUSTER FORGET messages to the cluster...

>>> 192.168.3.88:7399 as replica of 192.168.3.88:6399

/usr/lib/ruby/gems/1.8/gems/redis-3.0.7/lib/redis/client.rb:97:in `call': ERR Can't forget my master!(Redis::CommandError)

         from /usr/lib/ruby/gems/1.8/gems/redis-3.0.7/lib/redis.rb:2432:in `method_missing'

         from /usr/lib/ruby/gems/1.8/gems/redis-3.0.7/lib/redis.rb:37:in `synchronize'

         from /usr/lib/ruby/1.8/monitor.rb:242:in `mon_synchronize'

         from /usr/lib/ruby/gems/1.8/gems/redis-3.0.7/lib/redis.rb:37:in `synchronize'

         from /usr/lib/ruby/gems/1.8/gems/redis-3.0.7/lib/redis.rb:2431:in `method_missing'

         from /usr/local/bin/redis-trib.rb:1086:in `delnode_cluster_cmd'

         from /usr/local/bin/redis-trib.rb:1078:in `each'

         from /usr/local/bin/redis-trib.rb:1078:in `delnode_cluster_cmd'

         from /usr/local/bin/redis-trib.rb:1373:in `send'

         from /usr/local/bin/redis-trib.rb:1373

删除主节点之前，需要先删除主节点的slave节点，否则会报如上错误

# redis-trib.rb del-node 192.168.3.88:7399 d013aee7cae8163f787cb6445778ff97bf66ce17

>>> Removing node d013aee7cae8163f787cb6445778ff97bf66ce17 from cluster 192.168.3.88:7399

Connecting to node 192.168.3.88:7399: OK

Connecting to node 192.168.3.61:6379: OK

Connecting to node 192.168.3.61:7379: OK

Connecting to node 192.168.3.88:7379: OK

Connecting to node 192.168.3.88:6399: OK

Connecting to node 192.168.3.88:6379: OK

Connecting to node 192.168.3.88:7389: OK

Connecting to node 192.168.3.88:6389: OK

>>> Sending CLUSTER FORGET messages to the cluster...

>>> SHUTDOWN the node.

# redis-trib.rb del-node 192.168.3.88:6399 d64223d6695fcc7e1030f219f09d7488c438cf39

>>> Removing node d64223d6695fcc7e1030f219f09d7488c438cf39 from cluster 192.168.3.88:6399

Connecting to node 192.168.3.88:6399: OK

Connecting to node 192.168.3.61:6379: OK

Connecting to node 192.168.3.88:6379: OK

Connecting to node 192.168.3.88:7389: OK

Connecting to node 192.168.3.88:7379: OK

Connecting to node 192.168.3.88:6389: OK

Connecting to node 192.168.3.61:7379: OK

>>> Sending CLUSTER FORGET messages to the cluster...

>>> SHUTDOWN the node.