面试题 redis集群扩容 槽位迁移 redis集群扩容槽点丢失

redis集群官网文档

https://redis.io/topics/cluster-tutorial

 下载文档

https://redis.io/download

 

23种服务脚本
https://github.com/goodboy23/shell-script-collection 

yum install -y wget
wget http://download.redis.io/releases/redis-4.0.9.tar.gz
wget https://cache.ruby-lang.org/pub/ruby/2.5/ruby-2.5.1.tar.gz
wget https://rubygems.org/downloads/redis-4.0.0.gem

 

一 环境准备
1
gcc -v
gcc version 4.8.5 20150623 (Red Hat 4.8.5-16) (GCC)
yum -y install gcc gcc-c++

yum install -y gcc g++ make gcc-c++ kernel-devel automake autoconf libtool make wget tcl vim  unzip git

 

yum -y install zlib-devel #不编译安装后面会报错

yum -y install openssl-devel #不编译安装后面会报错
2.解压redis并编译安装
wget http://download.redis.io/releases/redis-4.0.9.tar.gz

mkdir /app/redis/ -p
tar -xvf redis-4.0.9.tar.gz -C /app/redis
cd /app/redis/redis-4.0.9/
make
make install

多了几个文件
ls /usr/local/bin

3.编译安装ruby

cd
tar -xvf ruby-2.5.1.tar.gz -C /app/redis
cd /app/redis/ruby-2.5.1/
./configure --prefix=/usr/local/ruby
make && make install

ls /usr/local/ruby #查看文件

配置ruby环境变量

vi /etc/profile

export PATH=$PATH:/usr/local/ruby/bin

:wq

source /etc/profile

echo $PATH
ruby -v

4.创建redis-cluster集群目录并拷贝redis的gem包以及在src下将redis-trib.rb 集群管理工具拷贝到集群目录

cd 
mkdir -p /app/redis/redis-cluster
cp redis-4.0.0.gem /app/redis/redis-cluster/
cp /app/redis/redis-4.0.9/src/redis-trib.rb /app/redis/redis-cluster/
cd /app/redis/redis-cluster/

 

5.找到原先ruby安装目录，并做如下修改

cd /app/redis/ruby-2.5.1/ext/zlib

ruby extconf.rb --成功会出现creating Makefile
creating Makefile --如果没有出现creating Makefile，执行下面的命令
ruby extconf.rb --with-zlib-dir=/usr/local/zlib/

vi Makefile

将 zlib.o: $(top_srcdir)/include/ruby.h
修改为 zlib.o: ../../include/ruby.h

 

make
make install

 

如果上面再执行make之前不修改Makefile,将会报下面的错误
make: * No rule to make target /include/ruby.h', needed byzlib.o’. Stop.

6.找到原先的ruby安装目录并做如下修改

cd /app/redis/ruby-2.5.1/ext/openssl
ruby extconf.rb --成功会出现creating Makefile

vi Makefile
将所有的$(top_srcdir)修改为 ../..（$(top_srcdir)不止一个）
:%s/vivian/sky/g（等同于 :g/vivian/s//sky/g） 替换每一行中所有 vivian 为 sky

:%s#$(top_srcdir)#../..#g #有31个

make
linking shared-object openssl.so
make install
/usr/bin/install -c -m 0755 openssl.so /usr/local/ruby/lib/ruby/site_ruby/2.5.0/x86_64-linux
installing default openssl libraries
如果上面再执行make之前不修改Makefile,将会报下面的错误
make: * No rule to make target /include/ruby.h', needed byossl.o’. Stop.

 

7.使用gem安装redis的gem包

cd /app/redis/redis-cluster
gem install redis-4.0.0.gem

 

 

二 创建集群

1.集群节点准备
cd /app/redis/redis-cluster/
mkdir 700{1,2,3,4,5,6}

cd /app/redis/redis-4.0.9
cp redis.conf /app/redis/redis-cluster/7001/
cp redis.conf /app/redis/redis-cluster/7002/
cp redis.conf /app/redis/redis-cluster/7003/
cp redis.conf /app/redis/redis-cluster/7004/
cp redis.conf /app/redis/redis-cluster/7005/
cp redis.conf /app/redis/redis-cluster/7006/

cd /usr/local/bin/
cp redis-server redis-cli /app/redis/redis-cluster/7001/
cp redis-server redis-cli /app/redis/redis-cluster/7002/
cp redis-server redis-cli /app/redis/redis-cluster/7003/
cp redis-server redis-cli /app/redis/redis-cluster/7004/
cp redis-server redis-cli /app/redis/redis-cluster/7005/
cp redis-server redis-cli /app/redis/redis-cluster/7006/

2.
/app/redis/redis-cluster/
修改7001-7006中的redis.conf
bind 192.168.1.62 连入主机的ip地址，不修改外部无法连入你的redis缓存服务器中
port 700X x为文件夹名称，你在700几就填几
daemonize yes 开启守护进程模式。在该模式下，redis会在后台运行，并将进程pid号写入至redis.conf选项pidfile设置的文件中，此时redis将一直运行，除非手动kill该进程。
pidfile /app/redis/redis-cluster/700x/redis_700x.pid x为文件夹名称，你在700几就填几
cluster-enabled yes 开启集群模式

vi /app/redis/redis-cluster/7001/redis.conf
cp /app/redis/redis-cluster/7001/redis.conf /app/redis/redis-cluster/7002/redis.conf
cp /app/redis/redis-cluster/7001/redis.conf /app/redis/redis-cluster/7003/redis.conf
cp /app/redis/redis-cluster/7001/redis.conf /app/redis/redis-cluster/7004/redis.conf
cp /app/redis/redis-cluster/7001/redis.conf /app/redis/redis-cluster/7005/redis.conf
cp /app/redis/redis-cluster/7001/redis.conf /app/redis/redis-cluster/7006/redis.conf

vi /app/redis/redis-cluster/7003/redis.conf
:%s/7001/7003/g

3.写批处理执行

cd /app/redis/redis-cluster

vi start-all.sh
cd 7001
./redis-server redis.conf
cd ..
cd 7002
./redis-server redis.conf
cd ..
cd 7003
./redis-server redis.conf
cd ..
cd 7004
./redis-server redis.conf
cd ..
cd 7005
./redis-server redis.conf
cd ..
cd 7006
./redis-server redis.conf
cd ..

开启命令执行权限
chmod +x start-all.sh

./start-all.sh
ps -ef|grep redis

4.
配置集群

# 在任意一台上运行即可，不要在每台机器上都运行

/app/redis/redis-cluster
创建了三个主节点，三个从节点。其中—replicas1 表示每个主节点下面有1个从节点，从节点可以是任意多个。前三个是主后三个是从

./redis-trib.rb create --replicas 1 192.168.1.62:7001 192.168.1.62:7002 192.168.1.62:7003 192.168.1.62:7004 192.168.1.62:7005 192.168.1.62:7006
yes

5集群测试
cd /app/redis/redis-cluster/7001
./redis-cli -c -h 192.168.1.62 -p 7001
./redis-cli -c -h 192.168.1.62 -p 7001 shutdown //关闭7001节点，如果没有-h参数，默认连接127.0.0.1，如果没有-p参数，默认连接6379端口（所有如果用默认的，就没有-h -p）
说明：-h+host -p+端口号 -c是要连接集群，注意坑，不加会报错的

root@node62 redis-cluster]# cd /app/redis/redis-cluster/7001
[root@node62 7001]# ./redis-cli -c -h 192.168.1.62 -p 7001
192.168.1.62:7001> set name xiaohei
-> Redirected to slot [5798] located at 192.168.1.62:7002
OK
192.168.1.62:7002> get name
"xiaohei"
192.168.1.62:7002> exit

[root@node62 7001]# ./redis-cli -c -h 192.168.1.62 -p 7001
192.168.1.62:7001> CLUSTER INFO

批量数据插入

vi pl.sh
#!/bin/bash
for ((i=0;i<1000;i++))
do
echo -en "helloworld" | /app/redis/redis-cluster/7001/redis-cli -c -h 192.168.1.62 -p 7001 -x set name$i >>redis.log
done

sh pl.sh

 

添加主从

指定redis cluster中的master和slave 可以用下面的方法

# 添加主库
[root@redis01 redis-cluster]# redis-trib.rb create 192.168.94.11:7000 192.168.94.22:7003 192.168.94.33:7006
# 添加从库
[root@redis01 redis-cluster]# redis-trib.rb add-node --slave 192.168.94.22:7001 192.168.94.11:7000    # 把192.168.94.11的7001 作为从库添加到192.168.94.11的7000

查看集群主从情况

[root@81server 7001]# ./redis-cli -c -h 192.168.120.81 -p 7001
192.168.120.81:7001> CLUSTER NODES

 

另一种遍历准备集群文件的方式

集群2安装
yum install -y gcc g++ make gcc-c++ kernel-devel automake autoconf libtool make wget tcl vim ruby rubygems unzip git
mkdir /app/redis/ -p
tar -xvf redis-4.0.9.tar.gz -C /app/redis
cd /app/redis/redis-4.0.9/
make
make && make test && make PREFIX=/usr/local/redis install
或者
make MALLOC=jemalloc & & make PREFIX=/usr/local/redis install

ls /usr/local/bin/


mkdir /data/redis/redis-cluster -p
cd /data/redis/redis-cluster/
mkdir 7000 7001 7002 

for i in 0 1 2;
 do echo -e "port 700$i\n\
 bind `hostname -I`\n\
 daemonize yes\n\
 pidfile /var/run/redis_700$i.pid\n\
 cluster-enabled yes\n\
 cluster-config-file nodes_700$i.conf\n\
 cluster-node-timeout 10100" > 700$i/redis.conf;
 done

for i in 0 1 2;
do redis-server /data/redis/redis-cluster/700$i/redis.conf 
done

yum install -y net-tools
netstat -antup|grep redis

[root@82server redis-cluster]# netstat -antup|grep redis
tcp        0      0 192.168.120.82:17000    0.0.0.0:*               LISTEN      21003/redis-server  
tcp        0      0 192.168.120.82:17001    0.0.0.0:*               LISTEN      21005/redis-server  
tcp        0      0 192.168.120.82:17002    0.0.0.0:*               LISTEN      21007/redis-server  
tcp        0      0 192.168.120.82:7000     0.0.0.0:*               LISTEN      21003/redis-server  
tcp        0      0 192.168.120.82:7001     0.0.0.0:*               LISTEN      21005/redis-server  
tcp        0      0 192.168.120.82:7002     0.0.0.0:*               LISTEN      21007/redis-server  
cd /app/redis/redis-cluster
./redis-trib.rb create 192.168.120.82:7001192.168.120.82:7000

./redis-trib.rb add-node --slave 192.168.120.82:7000 192.168.120.82:7001    # 把192.168.120.82:17001 作为从库添加到192.168.120.82:7001


./redis-trib.rb create 192.168.120.82:7000 192.168.120.82:7001


cd /app/redis/redis-cluster/

./redis-trib.rb create --replicas 1 192.168.1.62:7001 192.168.1.62:7002 192.168.1.62:7003 192.168.1.62:7004 192.168.1.62:7005 192.168.1.62:7006
yes

 

/app/redis/redis-cluster/7001/redis-cli -c -h 192.168.120.81 -p 7001 登录


/app/redis/redis-cluster/redis-trib.rb add-node 192.168.120.81:7004  192.168.120.81:7001 #前面是新节点 后面是集群内的节点 --添加新节点
/app/redis/redis-cluster/redis-trib.rb reshard 192.168.120.81:7001 #查看集群曹点和分配

 

 

 

 

 

 

 

 

 上面已经添加入了新的主节点但是没有曹点是无法存储数据的可以分配曹点

 

 

4个主节点平均分配一个4096个曹点

 

 后面复制一个新的集群id

然后输入all回车

 

 重新查看后已经平均分配

 

 加入新的从节点

/app/redis/redis-cluster/redis-trib.rb add-node 192.168.120.82:7004 192.168.120.81:7004 #加入新的节点
/app/redis/redis-cluster/7001/redis-cli  -p 7004 -h 192.168.120.82 #登录新加节点
cluster replicate 3bce7fac36c78b4f8a7db2371ef48ecbed458e44 #从主节id点同步数据 id可以用 CLUSTER NODES查询

 

 移除集群中的主节点操作

 

 

/app/redis/redis-cluster/redis-trib.rb del-node 192.168.120.82:7004 40c7b155fefe9b89c6e50e08786c7ebd95d7d935 #删除从节点
/app/redis/redis-cluster/redis-trib.rb reshard 192.168.120.81:7001 #移除主节点上的曹点
/app/redis/redis-cluster/redis-trib.rb del-node 192.168.120.81:7004 3bce7fac36c78b4f8a7db2371ef48ecbed458e44 #移除主节点

 

 

满血拉二胡 残血到处浪