一.redis集群的概念
现用redis做缓存服务器,redis是一个单点,当一台机器岩机的时候,redis的服务完全停止,这时就会影响其他服务的正常运行,所以我们要做的是redis主从复制及主动切换。
首先介绍下怎么样做redis主备切换,需要用到redis的sentinel做一个主从切换的集群管理。redis主从服务(1个master,多个salve),然后通过redis官方的监控工具Sentinel(哨兵),对每个节点进行监控,实现自动故障迁移,即master死掉,将salve升级为master。基本原理是:心跳机制+投票裁决。
Redis 的 Sentinel 系统用于管理多个 Redis 服务器(instance), 该系统执行以下三个任务:
监控(Monitoring): Sentinel 会不断地检查你的主服务器和从服务器是否运作正常。
提醒(Notification): 当被监控的某个 Redis 服务器出现问题时, Sentinel 可以通过API 向管理员或者其他应用程序发送通知。
自动故障迁移(Automatic failover): 当一个主服务器不能正常工作时,Sentinel 会开始一次自动故障迁移操作, 它会将失效主服务器的其中一个从服务器升级为新的主服务器, 并让失效主服务器的其他从服务器改为复制新的主服务器;当客户端试图连接失效的主服务器时, 集群也会向客户端返回新主服务器的地址, 使得集群可以使用新主服务器代替失效服务器。
Redis Sentinel 是一个分布式系统, 你可以在一个架构中运行多个 Sentinel 进程(progress), 这些进程使用流言协议(gossip protocols)来接收关于主服务器是否下线的信息, 并使用投票协议(agreementprotocols)来决定是否执行自动故障迁移,以及选择哪个从服务器作为新的主服务器。
虽然 Redis Sentinel 释出为一个单独的可执行文件 redis-sentinel, 但实际上它只是一个运行在特殊模式下的 Redis 服务器, 你可以在启动一个普通 Redis 服务器时通过给定 --sentinel 选项来启动Redis Sentinel 。
实验环境:er2:
主:server1:172.25.17.1
从:server2:172.25.17.2
实验部署:
配置主从复制:
主:
1.下载; http://download.redis.io/releases/redis-5.0.3.tar.gz
2.编译安装:
[root@server1 ~]#tar zxf redis-5.0.3.tar.zxf
[root@server1 ~]# cd redis-5.0.3/
[root@server1 ~]# systemctl stop mysqld
[root@server1 ~]# systemctl disabled mysqld
Unknown operation 'disabled'.
[root@server1 ~]# systemctl disable mysqld
Removed symlink /etc/systemd/system/multi-user.target.wants/mysqld.service.
[root@server1 ~]# cd redis-5.0.3
[root@server1 redis-5.0.3]# make && make install
cd src && make all
make[1]: Entering directory `/root/redis-5.0.3/src'
Hint: It's a good idea to run 'make test' ;)
make[1]: Leaving directory `/root/redis-5.0.3/src'
cd src && make install
make[1]: Entering directory `/root/redis-5.0.3/src'
Hint: It's a good idea to run 'make test' ;)
INSTALL install
INSTALL install
INSTALL install
INSTALL install
INSTALL install
make[1]: Leaving directory `/root/redis-5.0.3/src'
[root@server1 redis-5.0.3]# cd utils/
[root@server1 utils]# ./install_server.sh #默认回车
Welcome to the redis service installer
This script will help you easily set up a running redis server
Please select the redis port for this instance: [6379]
Selecting default: 6379
Please select the redis config file name [/etc/redis/6379.conf]
Selected default - /etc/redis/6379.conf
Please select the redis log file name [/var/log/redis_6379.log]
Selected default - /var/log/redis_6379.log
Please select the data directory for this instance [/var/lib/redis/6379]
Selected default - /var/lib/redis/6379
Please select the redis executable path [/usr/local/bin/redis-server]
Selected config:
Port : 6379
Config file : /etc/redis/6379.conf
Log file : /var/log/redis_6379.log
Data dir : /var/lib/redis/6379
Executable : /usr/local/bin/redis-server
Cli Executable : /usr/local/bin/redis-cli
Is this ok? Then press ENTER to go on or Ctrl-C to abort.
Copied /tmp/6379.conf => /etc/init.d/redis_6379
Installing service...
Successfully added to chkconfig!
Successfully added to runlevels 345!
/var/run/redis_6379.pid exists, process is already running or crashed
Installation successful!
[root@server1 utils]# /etc/init.d/redis_6379 status
Redis is running (892)
[root@server1 utils]# /etc/init.d/redis_6379 restart
Stopping ...
Redis stopped
Starting Redis server...
[root@server1 utils]# vim /etc/redis/6379.conf #监听访问ip修改,设置为监听本机所有6379端口
70 bind 0.0.0.0
[root@server1 utils]# /etc/init.d/redis_6379 restart
Stopping ...
Redis stopped
Starting Redis server...
[root@server1 utils]# netstat -antlp #查看端口为6379
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:6379 0.0.0.0:* LISTEN 2489/redis-server 0
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1366/sshd
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 2020/master
tcp 0 0 127.0.0.1:6379 127.0.0.1:60340 TIME_WAIT -
tcp 0 0 172.25.17.1:22 172.25.17.250:36060 ESTABLISHED 2131/sshd: root@pts
tcp 0 0 172.25.17.1:6379 172.25.17.4:39042 ESTABLISHED 2489/redis-server 0
tcp6 0 0 :::22 :::* LISTEN 1366/sshd
tcp6 0 0 ::1:25 :::* LISTEN 2020/master
从:
1.下载; http://download.redis.io/releases/redis-5.0.3.tar.gz
2.编译安装
[root@server2 ~]# tar zxf redis-5.0.3.tar.zxf
[root@server2 ~]# cd redis-5.0.3/
[root@server2 ~]# systemctl stop mysqld
[root@server2 ~]# systemctl disable mysqld
Removed symlink /etc/systemd/system/multi-user.target.wants/mysqld.service.
[root@server2 ~]# cd redis-5.0.3/
[root@server2 redis-5.0.3]# make && make install
cd src && make all
make[1]: Entering directory `/root/redis-5.0.3/src'
Hint: It's a good idea to run 'make test' ;)
make[1]: Leaving directory `/root/redis-5.0.3/src'
cd src && make install
make[1]: Entering directory `/root/redis-5.0.3/src'
Hint: It's a good idea to run 'make test' ;)
INSTALL install
INSTALL install
INSTALL install
INSTALL install
INSTALL install
make[1]: Leaving directory `/root/redis-5.0.3/src'
[root@server2 redis-5.0.3]# cd utils/
[root@server2 utils]# ./install_server.sh #默认回车
Welcome to the redis service installer
This script will help you easily set up a running redis server
Please select the redis port for this instance: [6379]
Selecting default: 6379
Please select the redis config file name [/etc/redis/6379.conf]
Selected default - /etc/redis/6379.conf
Please select the redis log file name [/var/log/redis_6379.log]
Selected default - /var/log/redis_6379.log
Please select the data directory for this instance [/var/lib/redis/6379]
Selected default - /var/lib/redis/6379
Please select the redis executable path [/usr/local/bin/redis-server]
Selected config:
Port : 6379
Config file : /etc/redis/6379.conf
Log file : /var/log/redis_6379.log
Data dir : /var/lib/redis/6379
Executable : /usr/local/bin/redis-server
Cli Executable : /usr/local/bin/redis-cli
Is this ok? Then press ENTER to go on or Ctrl-C to abort.
Copied /tmp/6379.conf => /etc/init.d/redis_6379
Installing service...
Successfully added to chkconfig!
Successfully added to runlevels 345!
/var/run/redis_6379.pid exists, process is already running or crashed
Installation successful!
[root@server2 utils]# /etc/init.d/redis_6379 status
Redis is running (1363)
[root@server2 utils]# /etc/init.d/redis_6379 restart
Stopping ...
Redis stopped
Starting Redis server...
[root@server2 utils]# vim /etc/redis/6379.conf #监听访问ip修改
70 bind 0.0.0.0
[root@server2 utils]# vim /etc/redis/6379.conf
文章末尾加入如下,指定连接那个库,这里设置为server1的ip以及接口:
slaveof 172.25.17.1 6379
[root@server2 utils]# /etc/init.d/redis_6379 restart
Stopping ...
Redis stopped
Starting Redis server...
[root@server2 utils]# netstat -antlp
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:6379 0.0.0.0:* LISTEN 2382/redis-server 0
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1855/sshd
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 2024/master
tcp 0 0 172.25.17.2:22 172.25.17.250:52914 ESTABLISHED 2135/sshd: root@pts
tcp 0 0 127.0.0.1:6379 127.0.0.1:41282 TIME_WAIT -
tcp6 0 0 :::22 :::* LISTEN 1855/sshd
tcp6 0 0 ::1:25 :::* LISTEN 2024/master
测试:
在server1上建立一条key-valye数据,server2上可以看到同步成功
[root@server1 utils]# redis-cli
127.0.0.1:6379> set xys 234
OK
[root@server2 utils]# redis-cli
127.0.0.1:6379> get xys
"234"
主从切换:
主从配置一致:
root@server1 ~]# cd redis-5.0.3
[root@server1 redis-5.0.3]# cp sentinel.conf /etc/redis/
[root@server1 redis-5.0.3]# vim /etc/redis/sentinel.conf
17 protected-mode no
18
84 sentinel monitor mymaster 172.25.17.1 6379 2
146 sentinel parallel-syncs mymaster 10000
121sentinel parallel-sync mymaster 1
启动
[root@server1 redis-5.0.3]# redis-server /etc/redis/sentinel.conf --sentinel
从机:server4:172.25.17.4
编译安装redis
[root@server1 redis-5.0.3]# yum install rsync -y
[root@server1 ~]# rsync -a redis-5.0.3 server4:
server4;
[root@server4 ~]# yum install rsync -y
[root@server4 ~]# cd redis-5.0.3/
[root@server4 redis-5.0.3]# make install
cd src && make install
make[1]: Entering directory `/root/redis-5.0.3/src'
Hint: It's a good idea to run 'make test' ;)
INSTALL install
INSTALL install
INSTALL install
INSTALL install
INSTALL install
make[1]: Leaving directory `/root/redis-5.0.3/src'
[root@server4 redis-5.0.3]# cd utils/
[root@server4 utils]# ./install_server.sh
Welcome to the redis service installer
This script will help you easily set up a running redis server
Please select the redis port for this instance: [6379]
Selecting default: 6379
Please select the redis config file name [/etc/redis/6379.conf]
Selected default - /etc/redis/6379.conf
Please select the redis log file name [/var/log/redis_6379.log]
Selected default - /var/log/redis_6379.log
Please select the data directory for this instance [/var/lib/redis/6379]
Selected default - /var/lib/redis/6379
Please select the redis executable path [/usr/local/bin/redis-server]
Selected config:
Port : 6379
Config file : /etc/redis/6379.conf
Log file : /var/log/redis_6379.log
Data dir : /var/lib/redis/6379
Executable : /usr/local/bin/redis-server
Cli Executable : /usr/local/bin/redis-cli
Is this ok? Then press ENTER to go on or Ctrl-C to abort.
Copied /tmp/6379.conf => /etc/init.d/redis_6379
Installing service...
Successfully added to chkconfig!
Successfully added to runlevels 345!
/var/run/redis_6379.pid exists, process is already running or crashed
Installation successful!
[root@server4 utils]# vim /etc/redis/6379.conf
[root@server4 utils]# /etc/init.d/redis_6379 restart
Stopping ...
Redis stopped
Starting Redis server...
[root@server4 utils]# cd
[root@server4 ~]# vim /etc/redis/6379.conf
[root@server4 ~]# /etc/init.d/redis_6379 restart
Stopping ...
Redis stopped
Starting Redis server...
[root@server4 ~]# redis-cli
127.0.0.1:6379> get name
(nil)
127.0.0.1:6379> get xys
"234"
127.0.0.1:6379> exit
server1:
[root@server1 redis]# scp sentinel.conf server2:/etc/redis/
sentinel.conf 100% 9711 9.5KB/s 00:00
[root@server1 redis]# scp sentinel.conf server4:/etc/redis/
sentinel.conf 100% 9711 9.5KB/s 00:00
[root@server1 redis]# redis-server /etc/redis/sentinel.conf --sentinel
2547:X 28 Feb 2019 11:34:27.248 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
2547:X 28 Feb 2019 11:34:27.248 # Redis version=5.0.3, bits=64, commit=00000000, modified=0, pid=2547, just started
2547:X 28 Feb 2019 11:34:27.248 # Configuration loaded
2547:X 28 Feb 2019 11:34:27.249 * Increased maximum number of open files to 10032 (it was originally set to 1024).
_._
_.-``__ ''-._
_.-`` `. `_. ''-._ Redis 5.0.3 (00000000/0) 64 bit
.-`` .-```. ```\/ _.,_ ''-._
( ' , .-` | `, ) Running in sentinel mode
|`-._`-...-` __...-.``-._|'` _.-'| Port: 26379
| `-._ `._ / _.-' | PID: 2547
`-._ `-._ `-./ _.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' | http://redis.io
`-._ `-._`-.__.-'_.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' |
`-._ `-._`-.__.-'_.-' _.-'
`-._ `-.__.-' _.-'
`-._ _.-'
`-.__.-'
2547:X 28 Feb 2019 11:34:27.249 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
2547:X 28 Feb 2019 11:34:27.311 # Sentinel ID is 7edf3c86b786f815d70ba471ab4b71f99d018dd4
2547:X 28 Feb 2019 11:34:27.311 # +monitor master mymaster 172.25.17.1 6379 quorum 2
2547:X 28 Feb 2019 11:34:27.312 * +slave slave 172.25.17.2:6379 172.25.17.2 6379 @ mymaster 172.25.17.1 6379
2547:X 28 Feb 2019 11:34:27.332 * +slave slave 172.25.17.4:6379 172.25.17.4 6379 @ mymaster 172.25.17.1 6379
2547:X 28 Feb 2019 11:35:11.441 * +sentinel sentinel 362b5bef2fcee02ae39b28e873828334a10c9346 172.25.17.2 26379 @ mymaster 172.25.17.1 6379
2547:X 28 Feb 2019 11:35:17.358 * +sentinel sentinel 8ec059feb2d8c2feec053ff3d81da2f1a4eef9e2 172.25.17.4 26379 @ mymaster 172.25.17.1 6379
$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
[root@server2 utils]# redis-server /etc/redis/sentinel.conf --sentinel
2431:X 28 Feb 2019 11:35:10.269 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
2431:X 28 Feb 2019 11:35:10.269 # Redis version=5.0.3, bits=64, commit=00000000, modified=0, pid=2431, just started
2431:X 28 Feb 2019 11:35:10.269 # Configuration loaded
2431:X 28 Feb 2019 11:35:10.270 * Increased maximum number of open files to 10032 (it was originally set to 1024).
_._
_.-``__ ''-._
_.-`` `. `_. ''-._ Redis 5.0.3 (00000000/0) 64 bit
.-`` .-```. ```\/ _.,_ ''-._
( ' , .-` | `, ) Running in sentinel mode
|`-._`-...-` __...-.``-._|'` _.-'| Port: 26379
| `-._ `._ / _.-' | PID: 2431
`-._ `-._ `-./ _.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' | http://redis.io
`-._ `-._`-.__.-'_.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' |
`-._ `-._`-.__.-'_.-' _.-'
`-._ `-.__.-' _.-'
`-._ _.-'
`-.__.-'
2431:X 28 Feb 2019 11:35:10.271 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
2431:X 28 Feb 2019 11:35:10.310 # Sentinel ID is 362b5bef2fcee02ae39b28e873828334a10c9346
2431:X 28 Feb 2019 11:35:10.310 # +monitor master mymaster 172.25.17.1 6379 quorum 2
2431:X 28 Feb 2019 11:35:10.312 * +slave slave 172.25.17.2:6379 172.25.17.2 6379 @ mymaster 172.25.17.1 6379
2431:X 28 Feb 2019 11:35:10.332 * +slave slave 172.25.17.4:6379 172.25.17.4 6379 @ mymaster 172.25.17.1 6379
2431:X 28 Feb 2019 11:35:10.764 * +sentinel sentinel 7edf3c86b786f815d70ba471ab4b71f99d018dd4 172.25.17.1 26379 @ mymaster 172.25.17.1 6379
2431:X 28 Feb 2019 11:35:18.235 * +sentinel sentinel 8ec059feb2d8c2feec053ff3d81da2f1a4eef9e2 172.25.17.4 26379 @ mymaster 172.25.17.1 6379
2431:X 28 Feb 2019 11:37:11.761 # +sdown sentinel 7edf3c86b786f815d70ba471ab4b71f99d018dd4 172.25.17.1 26379 @ mymaster 172.25.17.1 6379
2431:X 28 Feb 2019 11:40:40.595 # -sdown sentinel 7edf3c86b786f815d70ba471ab4b71f99d018dd4 172.25.17.1 26379 @ mymaster 172.25.17.1 6379
2431:X 28 Feb 2019 11:41:22.492 # +sdown sentinel 7edf3c86b786f815d70ba471ab4b71f99d018dd4 172.25.17.1 26379 @ mymaster 172.25.17.1 6379
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@2
sercer4:
[root@server4 ~]# redis-server /etc/redis/sentinel.conf --sentinel
2428:X 28 Feb 2019 11:35:15.831 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
2428:X 28 Feb 2019 11:35:15.831 # Redis version=5.0.3, bits=64, commit=00000000, modified=0, pid=2428, just started
2428:X 28 Feb 2019 11:35:15.831 # Configuration loaded
2428:X 28 Feb 2019 11:35:15.831 * Increased maximum number of open files to 10032 (it was originally set to 1024).
_._
_.-``__ ''-._
_.-`` `. `_. ''-._ Redis 5.0.3 (00000000/0) 64 bit
.-`` .-```. ```\/ _.,_ ''-._
( ' , .-` | `, ) Running in sentinel mode
|`-._`-...-` __...-.``-._|'` _.-'| Port: 26379
| `-._ `._ / _.-' | PID: 2428
`-._ `-._ `-./ _.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' | http://redis.io
`-._ `-._`-.__.-'_.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' |
`-._ `-._`-.__.-'_.-' _.-'
`-._ `-.__.-' _.-'
`-._ _.-'
`-.__.-'
2428:X 28 Feb 2019 11:35:15.832 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
2428:X 28 Feb 2019 11:35:15.967 # Sentinel ID is 8ec059feb2d8c2feec053ff3d81da2f1a4eef9e2
2428:X 28 Feb 2019 11:35:15.968 # +monitor master mymaster 172.25.17.1 6379 quorum 2
2428:X 28 Feb 2019 11:35:15.969 * +slave slave 172.25.17.2:6379 172.25.17.2 6379 @ mymaster 172.25.17.1 6379
2428:X 28 Feb 2019 11:35:16.023 * +slave slave 172.25.17.4:6379 172.25.17.4 6379 @ mymaster 172.25.17.1 6379
2428:X 28 Feb 2019 11:35:16.082 * +sentinel sentinel 362b5bef2fcee02ae39b28e873828334a10c9346 172.25.17.2 26379 @ mymaster 172.25.17.1 6379
2428:X 28 Feb 2019 11:35:16.566 * +sentinel sentinel 7edf3c86b786f815d70ba471ab4b71f99d018dd4 172.25.17.1 26379 @ mymaster 172.25.17.1 6379
2428:X 28 Feb 2019 11:37:11.465 # +sdown sentinel 7edf3c86b786f815d70ba471ab4b71f99d018dd4 172.25.17.1 26379 @ mymaster 172.25.17.1 6379
2428:X 28 Feb 2019 11:40:40.426 # -sdown sentinel 7edf3c86b786f815d70ba471ab4b71f99d018dd4 172.25.17.1 26379 @ mymaster 172.25.17.1 6379
2428:X 28 Feb 2019 11:41:22.270 # +sdown sentinel 7edf3c86b786f815d70ba471ab4b71f99d018dd4 172.25.17.1 26379 @ mymaster 172.25.17.1 6379
由监控信息可见,此时master为172.25.17.1 两个slave为172.25.17.2和172.25.17.4
由监控信息可见,此时master为172.25.17.1 两个slave为172.25.17.2和172.25.17.4
sever1查询信息:
[root@server1 ~]# redis-cli -p 26379
127.0.0.1:26379> info
master0:name=mymaster,status=ok,address=172.25.17.1:6379,slaves=2,sentinels=3
此时挂掉server1:
redis-cli -p 6379
SHUTDOWN
server2和server4显示master已经交换到server2 此时的两个slave为172.25.17.1和172.25.17.4
sever2:
2431:X 28 Feb 2019 11:42:49.340 # +switch-master mymaster 172.25.17.1 6379 172.25.17.2 6379
2431:X 28 Feb 2019 11:42:49.340 * +slave slave 172.25.17.4:6379 172.25.17.4 6379 @ mymaster 172.25.17.2 6379
2431:X 28 Feb 2019 11:42:49.340 * +slave slave 172.25.17.1:6379 172.25.17.1 6379 @ mymaster 172.25.17.2 6379
2431:X 28 Feb 2019 11:45:50.230 * +fix-slave-config slave 172.25.17.1:6379 172.25.17.1 6379 @ mymaster 172.25.17.2 6379
server4:
2428:X 28 Feb 2019 11:42:50.057 # +switch-master mymaster 172.25.17.1 6379 172.25.17.2 6379
2428:X 28 Feb 2019 11:42:50.058 * +slave slave 172.25.17.4:6379 172.25.17.4 6379 @ mymaster 172.25.17.2 6379
2428:X 28 Feb 2019 11:42:50.058 * +slave slave 172.25.17.1:6379 172.25.17.1 6379 @ mymaster 172.25.17.2 6379
重新启动server1:
cd /etc/redis/
vim 6379.conf
写入:
slaveof 172.25.17.2 6379
/ect/init.d/redis_6379 status
/ect/init.d/redis_6379 start